[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:49:28 dist-test-slave-x5fp krb5kdc[3374](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:49:28 dist-test-slave-x5fp krb5kdc[3374](info): set up 2 sockets
Aug 07 10:49:28 dist-test-slave-x5fp krb5kdc[3374](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 10:49:29.422569 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.487s user 0.006s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:49:29 dist-test-slave-x5fp krb5kdc[3374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:49:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:49:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.73.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
I20260807 10:49:29.705422 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:43411
--webserver_interface=127.3.73.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:34631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:43411
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:49:30.409198 3396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:49:30.409742 3396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:49:30.410058 3396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:49:30.420905 3396 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:49:30.421130 3396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:49:30.421339 3396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:49:30.421541 3396 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:49:30.421684 3396 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:49:30.435899 3396 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:34631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:43411
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:43411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:49:30.438308 3396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:49:30.440683 3396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:49:30.466702 3401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:30.467293 3405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:30.469776 3402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:49:30.469797 3396 server_base.cc:1034] running on GCE node
I20260807 10:49:30.471114 3396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:49:30.473613 3396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:49:30.477104 3396 hybrid_clock.cc:648] HybridClock initialized: now 1786099770476932 us; error 154 us; skew 500 ppm
I20260807 10:49:30.477900 3396 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:49:30 dist-test-slave-x5fp krb5kdc[3374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099770, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:49:30.487498 3396 init.cc:377] Logged in from keytab as kudu/127.3.73.254@KRBTEST.COM (short username kudu)
I20260807 10:49:30.491307 3396 webserver.cc:460] Webserver started at http://127.3.73.254:36685/ using document root <none> and password file <none>
I20260807 10:49:30.492547 3396 fs_manager.cc:359] Metadata directory not provided
I20260807 10:49:30.492723 3396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:49:30.493350 3396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:49:30.498986 3396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "3da934ddae8949f0ab2722f869252faf"
format_stamp: "Formatted at 2026-08-07 10:49:30 on dist-test-slave-x5fp"
server_key: "9e5ce27f1f26c2b7b9c0029de4037448"
server_key_iv: "8805a598674304f68de475e28df2d93c"
server_key_version: "encryptionkey@0"
I20260807 10:49:30.500156 3396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "3da934ddae8949f0ab2722f869252faf"
format_stamp: "Formatted at 2026-08-07 10:49:30 on dist-test-slave-x5fp"
server_key: "9e5ce27f1f26c2b7b9c0029de4037448"
server_key_iv: "8805a598674304f68de475e28df2d93c"
server_key_version: "encryptionkey@0"
I20260807 10:49:30.511051 3396 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.005s sys 0.002s
I20260807 10:49:30.518708 3416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:30.521246 3396 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.003s sys 0.002s
I20260807 10:49:30.521620 3396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "3da934ddae8949f0ab2722f869252faf"
format_stamp: "Formatted at 2026-08-07 10:49:30 on dist-test-slave-x5fp"
server_key: "9e5ce27f1f26c2b7b9c0029de4037448"
server_key_iv: "8805a598674304f68de475e28df2d93c"
server_key_version: "encryptionkey@0"
I20260807 10:49:30.521971 3396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:49:30.574360 3396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:49:30.581956 3396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:49:30.582585 3396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:49:30.641745 3396 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:43411
I20260807 10:49:30.641851 3493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:43411 every 8 connection(s)
I20260807 10:49:30.644505 3396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:49:30.647745 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 3396
I20260807 10:49:30.648218 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:49:30.651309 3367 external_mini_cluster.cc:1428] Setting key b476c855350ce89d93ea28b7ce295e62
I20260807 10:49:30.653772 3494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:49:30.676852 3494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3da934ddae8949f0ab2722f869252faf: Bootstrap starting.
I20260807 10:49:30.684170 3494 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3da934ddae8949f0ab2722f869252faf: Neither blocks nor log segments found. Creating new log.
I20260807 10:49:30.687125 3494 log.cc:824] T 00000000000000000000000000000000 P 3da934ddae8949f0ab2722f869252faf: Log is configured to *not* fsync() on all Append() calls
I20260807 10:49:30.693073 3494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3da934ddae8949f0ab2722f869252faf: No bootstrap required, opened a new log
Aug 07 10:49:30 dist-test-slave-x5fp krb5kdc[3374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:49:30.707677 3494 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3da934ddae8949f0ab2722f869252faf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3da934ddae8949f0ab2722f869252faf" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 43411 } }
I20260807 10:49:30.708379 3494 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3da934ddae8949f0ab2722f869252faf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:49:30.708688 3494 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3da934ddae8949f0ab2722f869252faf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3da934ddae8949f0ab2722f869252faf, State: Initialized, Role: FOLLOWER
I20260807 10:49:30.710148 3494 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3da934ddae8949f0ab2722f869252faf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3da934ddae8949f0ab2722f869252faf" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 43411 } }
I20260807 10:49:30.710608 3494 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3da934ddae8949f0ab2722f869252faf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:49:30.710973 3494 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3da934ddae8949f0ab2722f869252faf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:49:30.711319 3494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3da934ddae8949f0ab2722f869252faf [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:49:30.715588 3497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:30.658859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45892 (local address 127.3.73.254:43411)
0807 10:49:30.661452 (+ 2593us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:30.661476 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:30.662980 (+ 1504us) server_negotiation.cc:407] Connection header received
0807 10:49:30.679490 (+ 16510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:30.679544 (+ 54us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:30.680232 (+ 688us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:30.680851 (+ 619us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:30.686134 (+ 5283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:30.688951 (+ 2817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:30.691427 (+ 2476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:30.692437 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:30.692775 (+ 338us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:30.701731 (+ 8956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:30.701788 (+ 57us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:30.701824 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:30.701878 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:30.707432 (+ 5554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:30.708834 (+ 1402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:30.708842 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:30.708855 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:30.709194 (+ 339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:30.710021 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:30.710029 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:30.710034 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:30.711069 (+ 1035us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:30.711660 (+ 591us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:30.712884 (+ 1224us) server_negotiation.cc:299] Negotiation successful
0807 10:49:30.713543 (+ 659us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1975,"thread_start_us":1417,"threads_started":1}
I20260807 10:49:30.715843 3494 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3da934ddae8949f0ab2722f869252faf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3da934ddae8949f0ab2722f869252faf" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 43411 } }
I20260807 10:49:30.717296 3494 leader_election.cc:302] T 00000000000000000000000000000000 P 3da934ddae8949f0ab2722f869252faf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3da934ddae8949f0ab2722f869252faf; no voters:
I20260807 10:49:30.719105 3494 leader_election.cc:288] T 00000000000000000000000000000000 P 3da934ddae8949f0ab2722f869252faf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:49:30.719383 3500 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3da934ddae8949f0ab2722f869252faf [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:49:30.724164 3494 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3da934ddae8949f0ab2722f869252faf [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:49:30.725180 3500 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3da934ddae8949f0ab2722f869252faf [term 1 LEADER]: Becoming Leader. State: Replica: 3da934ddae8949f0ab2722f869252faf, State: Running, Role: LEADER
I20260807 10:49:30.726711 3500 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3da934ddae8949f0ab2722f869252faf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3da934ddae8949f0ab2722f869252faf" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 43411 } }
I20260807 10:49:30.737063 3502 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3da934ddae8949f0ab2722f869252faf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3da934ddae8949f0ab2722f869252faf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3da934ddae8949f0ab2722f869252faf" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 43411 } } }
I20260807 10:49:30.737418 3503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3da934ddae8949f0ab2722f869252faf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3da934ddae8949f0ab2722f869252faf. Latest consensus state: current_term: 1 leader_uuid: "3da934ddae8949f0ab2722f869252faf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3da934ddae8949f0ab2722f869252faf" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 43411 } } }
I20260807 10:49:30.739020 3503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3da934ddae8949f0ab2722f869252faf [sys.catalog]: This master's current role is: LEADER
I20260807 10:49:30.739387 3502 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3da934ddae8949f0ab2722f869252faf [sys.catalog]: This master's current role is: LEADER
I20260807 10:49:30.750876 3515 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:49:30.762444 3515 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:49:30.784152 3515 catalog_manager.cc:1347] Generated new cluster ID: fff52d9045734a3791a7a32b2d027ea9
I20260807 10:49:30.784409 3515 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:49:30.805354 3515 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:49:30.807637 3515 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:49:30.820947 3515 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3da934ddae8949f0ab2722f869252faf: Generated new TSK 0
I20260807 10:49:30.822541 3515 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.73.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab.
Entry for principal kudu/127.3.73.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab.
I20260807 10:49:30.898427 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.193:0
--local_ip_for_outbound_sockets=127.3.73.193
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:43411
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:34631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:49:31.572382 3529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:49:31.572906 3529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:49:31.573148 3529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:49:31.582919 3529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:49:31.583173 3529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:49:31.583427 3529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.193
I20260807 10:49:31.596655 3529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:34631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:43411
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.193
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:49:31.598344 3529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:49:31.599996 3529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:49:31.619864 3537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:31.620069 3540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:31.620205 3538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:49:31.621735 3529 server_base.cc:1034] running on GCE node
I20260807 10:49:31.622530 3529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:49:31.623584 3529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:49:31.624856 3529 hybrid_clock.cc:648] HybridClock initialized: now 1786099771624790 us; error 45 us; skew 500 ppm
I20260807 10:49:31.625284 3529 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:49:31 dist-test-slave-x5fp krb5kdc[3374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099771, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:49:31.630630 3529 init.cc:377] Logged in from keytab as kudu/127.3.73.193@KRBTEST.COM (short username kudu)
I20260807 10:49:31.632892 3529 webserver.cc:460] Webserver started at http://127.3.73.193:45435/ using document root <none> and password file <none>
I20260807 10:49:31.633783 3529 fs_manager.cc:359] Metadata directory not provided
I20260807 10:49:31.634032 3529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:49:31.634493 3529 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:49:31.638532 3529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-0/data/instance:
uuid: "f65397cdc02447e1be20788d17b00079"
format_stamp: "Formatted at 2026-08-07 10:49:31 on dist-test-slave-x5fp"
server_key: "1d3f73db58e4c55c66296f9ad784141d"
server_key_iv: "7c40007362feea19b2a5ef28cc3583d3"
server_key_version: "encryptionkey@0"
I20260807 10:49:31.639756 3529 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance:
uuid: "f65397cdc02447e1be20788d17b00079"
format_stamp: "Formatted at 2026-08-07 10:49:31 on dist-test-slave-x5fp"
server_key: "1d3f73db58e4c55c66296f9ad784141d"
server_key_iv: "7c40007362feea19b2a5ef28cc3583d3"
server_key_version: "encryptionkey@0"
I20260807 10:49:31.646961 3529 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260807 10:49:31.652334 3550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:31.654008 3529 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 10:49:31.654287 3529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-0/wal
uuid: "f65397cdc02447e1be20788d17b00079"
format_stamp: "Formatted at 2026-08-07 10:49:31 on dist-test-slave-x5fp"
server_key: "1d3f73db58e4c55c66296f9ad784141d"
server_key_iv: "7c40007362feea19b2a5ef28cc3583d3"
server_key_version: "encryptionkey@0"
I20260807 10:49:31.654572 3529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:49:31.674463 3529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:49:31.680990 3529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:49:31.681550 3529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:49:31.683256 3529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:49:31.685613 3529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:49:31.685835 3529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:31.686097 3529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:49:31.686231 3529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:31.760782 3529 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.193:34633
I20260807 10:49:31.760919 3716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.193:34633 every 8 connection(s)
I20260807 10:49:31.762292 3529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
I20260807 10:49:31.766007 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 3529
I20260807 10:49:31.766307 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance
I20260807 10:49:31.767391 3367 external_mini_cluster.cc:1428] Setting key 371559f172ceef764c0345b0fdae3e37
Aug 07 10:49:31 dist-test-slave-x5fp krb5kdc[3374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099771, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:49:31.794124 3720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:31.766666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:34967 (local address 127.3.73.254:43411)
0807 10:49:31.767582 (+ 916us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:31.767589 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:31.769617 (+ 2028us) server_negotiation.cc:407] Connection header received
0807 10:49:31.772566 (+ 2949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:31.772574 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:31.772766 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:31.773187 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:31.775181 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:31.777192 (+ 2011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:31.780176 (+ 2984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:31.781414 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:31.781605 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:31.786104 (+ 4499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:31.786114 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:31.786122 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:31.786180 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:31.789910 (+ 3730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:31.791026 (+ 1116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:31.791031 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:31.791033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:31.791194 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:31.791786 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:31.791790 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:31.791792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:31.792115 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:31.792241 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:31.793528 (+ 1287us) server_negotiation.cc:299] Negotiation successful
0807 10:49:31.793741 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":739,"thread_start_us":478,"threads_started":1}
I20260807 10:49:31.795925 3721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:31.767313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:43411 (local address 127.3.73.193:34967)
0807 10:49:31.769159 (+ 1846us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:31.769224 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:31.772152 (+ 2928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:31.773470 (+ 1318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:31.773479 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:31.774194 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:49:31.774914 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:31.774937 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:31.777397 (+ 2460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:31.777401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:31.779987 (+ 2586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:31.780001 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:31.781704 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:31.781710 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:31.781970 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:31.782888 (+ 918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:49:31.782916 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:49:31.785731 (+ 2815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:49:31.790086 (+ 4355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:31.790099 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:31.790121 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:31.790892 (+ 771us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:31.791349 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:31.791357 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:31.791364 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:31.791657 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:31.792263 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:49:31.792275 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:49:31.792847 (+ 572us) client_negotiation.cc:769] Sending connection context
0807 10:49:31.793408 (+ 561us) client_negotiation.cc:240] Negotiation successful
0807 10:49:31.794075 (+ 667us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1009,"thread_start_us":623,"threads_started":1}
I20260807 10:49:31.798388 3718 heartbeater.cc:347] Connected to a master server at 127.3.73.254:43411
I20260807 10:49:31.798954 3718 heartbeater.cc:464] Registering TS with master...
I20260807 10:49:31.800096 3718 heartbeater.cc:511] Master 127.3.73.254:43411 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.3.73.194@KRBTEST.COM; defaulting to no policy
I20260807 10:49:31.804495 3442 ts_manager.cc:194] Registered new tserver with Master: f65397cdc02447e1be20788d17b00079 (127.3.73.193:34633)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.194@KRBTEST.COM" created.
I20260807 10:49:31.806932 3442 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.193@KRBTEST.COM'} at 127.3.73.193:34967
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab.
Entry for principal kudu/127.3.73.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab.
I20260807 10:49:31.844012 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.194:0
--local_ip_for_outbound_sockets=127.3.73.194
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:43411
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:34631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:49:32.499972 3724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:49:32.500425 3724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:49:32.500697 3724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:49:32.511147 3724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:49:32.511381 3724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:49:32.511684 3724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.194
I20260807 10:49:32.525341 3724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:34631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:43411
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.194
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:49:32.527470 3724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:49:32.529484 3724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:49:32.550530 3730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:32.550992 3732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:32.551864 3736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:49:32.554272 3724 server_base.cc:1034] running on GCE node
I20260807 10:49:32.555229 3724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:49:32.556418 3724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:49:32.557710 3724 hybrid_clock.cc:648] HybridClock initialized: now 1786099772557648 us; error 120 us; skew 500 ppm
I20260807 10:49:32.558193 3724 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:49:32 dist-test-slave-x5fp krb5kdc[3374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099772, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:49:32.563661 3724 init.cc:377] Logged in from keytab as kudu/127.3.73.194@KRBTEST.COM (short username kudu)
I20260807 10:49:32.566632 3724 webserver.cc:460] Webserver started at http://127.3.73.194:39749/ using document root <none> and password file <none>
I20260807 10:49:32.567643 3724 fs_manager.cc:359] Metadata directory not provided
I20260807 10:49:32.567914 3724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:49:32.568459 3724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:49:32.572976 3724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-1/data/instance:
uuid: "8440d46b2b224d91addaa6310690507b"
format_stamp: "Formatted at 2026-08-07 10:49:32 on dist-test-slave-x5fp"
server_key: "76072ca6b448cb12a86e6c2cbad02f1a"
server_key_iv: "8a7d2addfd87d3c661491da053a12ad3"
server_key_version: "encryptionkey@0"
I20260807 10:49:32.574082 3724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance:
uuid: "8440d46b2b224d91addaa6310690507b"
format_stamp: "Formatted at 2026-08-07 10:49:32 on dist-test-slave-x5fp"
server_key: "76072ca6b448cb12a86e6c2cbad02f1a"
server_key_iv: "8a7d2addfd87d3c661491da053a12ad3"
server_key_version: "encryptionkey@0"
I20260807 10:49:32.581854 3724 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.002s
I20260807 10:49:32.587706 3748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:32.589504 3724 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 10:49:32.589780 3724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-1/wal
uuid: "8440d46b2b224d91addaa6310690507b"
format_stamp: "Formatted at 2026-08-07 10:49:32 on dist-test-slave-x5fp"
server_key: "76072ca6b448cb12a86e6c2cbad02f1a"
server_key_iv: "8a7d2addfd87d3c661491da053a12ad3"
server_key_version: "encryptionkey@0"
I20260807 10:49:32.590106 3724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:49:32.619150 3724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:49:32.625500 3724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:49:32.626084 3724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:49:32.627979 3724 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:49:32.630259 3724 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:49:32.630442 3724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:32.630690 3724 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:49:32.630854 3724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:32.728684 3724 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.194:46075
I20260807 10:49:32.728938 3924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.194:46075 every 8 connection(s)
I20260807 10:49:32.730578 3724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
I20260807 10:49:32.736238 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 3724
I20260807 10:49:32.736552 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance
I20260807 10:49:32.737119 3367 external_mini_cluster.cc:1428] Setting key 5c2d068c9e62e1388244460690fa0530
Aug 07 10:49:32 dist-test-slave-x5fp krb5kdc[3374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099772, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:49:32.764127 3927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:32.734642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:39103 (local address 127.3.73.254:43411)
0807 10:49:32.735448 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:32.735453 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:32.737770 (+ 2317us) server_negotiation.cc:407] Connection header received
0807 10:49:32.741469 (+ 3699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:32.741474 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:32.741572 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:32.741865 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:32.743881 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:32.744964 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:32.746923 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:32.747627 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:32.747825 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:32.753337 (+ 5512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:32.753347 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:32.753354 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:32.753424 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:32.758826 (+ 5402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:32.760270 (+ 1444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:32.760279 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:32.760285 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:32.760483 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:32.761292 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:32.761300 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:32.761303 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:32.761814 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:32.761978 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:32.763042 (+ 1064us) server_negotiation.cc:299] Negotiation successful
0807 10:49:32.763385 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":521,"threads_started":1}
I20260807 10:49:32.765681 3928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:32.735545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:43411 (local address 127.3.73.194:39103)
0807 10:49:32.737279 (+ 1734us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:32.737330 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:32.741082 (+ 3752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:32.742235 (+ 1153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:32.742245 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:32.742979 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:49:32.743668 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:32.743687 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:32.745142 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:32.745148 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:32.746706 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:32.746720 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:32.748059 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:32.748078 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:32.748366 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:32.749232 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:49:32.749280 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:49:32.752842 (+ 3562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:49:32.759068 (+ 6226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:32.759080 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:32.759101 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:32.760053 (+ 952us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:32.760664 (+ 611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:32.760669 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:32.760672 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:32.761122 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:32.761978 (+ 856us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:49:32.761986 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:49:32.762478 (+ 492us) client_negotiation.cc:769] Sending connection context
0807 10:49:32.762884 (+ 406us) client_negotiation.cc:240] Negotiation successful
0807 10:49:32.763737 (+ 853us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1397,"thread_start_us":1087,"threads_started":1}
I20260807 10:49:32.768083 3925 heartbeater.cc:347] Connected to a master server at 127.3.73.254:43411
I20260807 10:49:32.768492 3925 heartbeater.cc:464] Registering TS with master...
I20260807 10:49:32.769518 3925 heartbeater.cc:511] Master 127.3.73.254:43411 requested a full tablet report, sending...
I20260807 10:49:32.771538 3442 ts_manager.cc:194] Registered new tserver with Master: 8440d46b2b224d91addaa6310690507b (127.3.73.194:46075)
I20260807 10:49:32.773079 3442 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.194@KRBTEST.COM'} at 127.3.73.194:39103
WARNING: no policy specified for kudu/127.3.73.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab.
Entry for principal kudu/127.3.73.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab.
I20260807 10:49:32.812173 3718 heartbeater.cc:503] Master 127.3.73.254:43411 was elected leader, sending a full tablet report...
I20260807 10:49:32.812984 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.195:0
--local_ip_for_outbound_sockets=127.3.73.195
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:43411
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:34631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:49:33.380097 3931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:49:33.380530 3931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:49:33.380806 3931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:49:33.390878 3931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:49:33.391105 3931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:49:33.391316 3931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.195
I20260807 10:49:33.403630 3931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:34631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:43411
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.195
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:49:33.405380 3931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:49:33.407176 3931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:49:33.426652 3938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:33.426786 3940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:33.426817 3943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:49:33.428609 3931 server_base.cc:1034] running on GCE node
I20260807 10:49:33.429388 3931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:49:33.430541 3931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:49:33.431844 3931 hybrid_clock.cc:648] HybridClock initialized: now 1786099773431759 us; error 82 us; skew 500 ppm
I20260807 10:49:33.432262 3931 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:49:33 dist-test-slave-x5fp krb5kdc[3374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099773, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:49:33.436789 3931 init.cc:377] Logged in from keytab as kudu/127.3.73.195@KRBTEST.COM (short username kudu)
I20260807 10:49:33.439230 3931 webserver.cc:460] Webserver started at http://127.3.73.195:40969/ using document root <none> and password file <none>
I20260807 10:49:33.440157 3931 fs_manager.cc:359] Metadata directory not provided
I20260807 10:49:33.440351 3931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:49:33.440814 3931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:49:33.444362 3931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-2/data/instance:
uuid: "4321514acbb64369ad723966884502d4"
format_stamp: "Formatted at 2026-08-07 10:49:33 on dist-test-slave-x5fp"
server_key: "394756fd4a8b12e2d858cdcf1f5318f8"
server_key_iv: "1d10b78b2ec569c29783999a17712c92"
server_key_version: "encryptionkey@0"
I20260807 10:49:33.445281 3931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance:
uuid: "4321514acbb64369ad723966884502d4"
format_stamp: "Formatted at 2026-08-07 10:49:33 on dist-test-slave-x5fp"
server_key: "394756fd4a8b12e2d858cdcf1f5318f8"
server_key_iv: "1d10b78b2ec569c29783999a17712c92"
server_key_version: "encryptionkey@0"
I20260807 10:49:33.451931 3931 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.003s
I20260807 10:49:33.456425 3955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:33.458479 3931 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 10:49:33.458752 3931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-2/wal
uuid: "4321514acbb64369ad723966884502d4"
format_stamp: "Formatted at 2026-08-07 10:49:33 on dist-test-slave-x5fp"
server_key: "394756fd4a8b12e2d858cdcf1f5318f8"
server_key_iv: "1d10b78b2ec569c29783999a17712c92"
server_key_version: "encryptionkey@0"
I20260807 10:49:33.459148 3931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:49:33.506032 3931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:49:33.511874 3931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:49:33.512235 3931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:49:33.513762 3931 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:49:33.515879 3931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:49:33.516012 3931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:33.516222 3931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:49:33.516330 3931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:33.592615 3931 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.195:34379
I20260807 10:49:33.592747 4114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.195:34379 every 8 connection(s)
I20260807 10:49:33.594383 3931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
I20260807 10:49:33.598158 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 3931
I20260807 10:49:33.598464 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance
I20260807 10:49:33.599177 3367 external_mini_cluster.cc:1428] Setting key 136d7cd760a138c8f272e7e5357932d2
Aug 07 10:49:33 dist-test-slave-x5fp krb5kdc[3374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099773, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:49:33.622450 4119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:33.598230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:42039 (local address 127.3.73.254:43411)
0807 10:49:33.598984 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:33.598990 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:33.600522 (+ 1532us) server_negotiation.cc:407] Connection header received
0807 10:49:33.603848 (+ 3326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:33.603852 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:33.603968 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:33.604174 (+ 206us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:33.606361 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:33.607232 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:33.609276 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:33.609888 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:33.609995 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:33.614632 (+ 4637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:33.614638 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:33.614642 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:33.614680 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:33.618770 (+ 4090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:33.619662 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:33.619667 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:33.619669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:33.619797 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:33.620280 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:33.620285 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:33.620287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:33.620633 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:33.620899 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:33.621825 (+ 926us) server_negotiation.cc:299] Negotiation successful
0807 10:49:33.622014 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":425,"threads_started":1}
I20260807 10:49:33.623322 4120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:33.598996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:43411 (local address 127.3.73.195:42039)
0807 10:49:33.600194 (+ 1198us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:33.600276 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:33.603506 (+ 3230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:33.604548 (+ 1042us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:33.604564 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:33.605383 (+ 819us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:49:33.606141 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:33.606161 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:33.607428 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:33.607433 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:33.609102 (+ 1669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:33.609116 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:33.610212 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:33.610219 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:33.610512 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:33.611415 (+ 903us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:49:33.611440 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:49:33.614335 (+ 2895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:49:33.618951 (+ 4616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:33.618959 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:33.618974 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:33.619529 (+ 555us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:33.619899 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:33.619903 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:33.619905 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:33.620135 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:33.620985 (+ 850us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:49:33.620992 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:49:33.621379 (+ 387us) client_negotiation.cc:769] Sending connection context
0807 10:49:33.621636 (+ 257us) client_negotiation.cc:240] Negotiation successful
0807 10:49:33.622096 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":840,"thread_start_us":591,"threads_started":1}
I20260807 10:49:33.625253 4117 heartbeater.cc:347] Connected to a master server at 127.3.73.254:43411
I20260807 10:49:33.625679 4117 heartbeater.cc:464] Registering TS with master...
I20260807 10:49:33.626580 4117 heartbeater.cc:511] Master 127.3.73.254:43411 requested a full tablet report, sending...
I20260807 10:49:33.628422 3442 ts_manager.cc:194] Registered new tserver with Master: 4321514acbb64369ad723966884502d4 (127.3.73.195:34379)
I20260807 10:49:33.629712 3442 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.195@KRBTEST.COM'} at 127.3.73.195:42039
I20260807 10:49:33.630996 3367 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 10:49:33 dist-test-slave-x5fp krb5kdc[3374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099773, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 10:49:33 dist-test-slave-x5fp krb5kdc[3374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099773, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.73.193@KRBTEST.COM
I20260807 10:49:33.707281 4129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:33.683803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49252 (local address 127.3.73.193:34633)
0807 10:49:33.684868 (+ 1065us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:33.684881 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:33.684920 (+ 39us) server_negotiation.cc:407] Connection header received
0807 10:49:33.685006 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:33.685012 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:33.685246 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:33.685456 (+ 210us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:33.687527 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:33.688578 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:33.690383 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:33.691242 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:33.691419 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:33.697196 (+ 5777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:33.697221 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:33.697254 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:33.697333 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:33.703139 (+ 5806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:33.704183 (+ 1044us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:33.704195 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:33.704200 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:33.704431 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:33.704928 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:33.704936 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:33.704942 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:33.705620 (+ 678us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:33.706022 (+ 402us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:33.706311 (+ 289us) server_negotiation.cc:299] Negotiation successful
0807 10:49:33.706743 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":853,"thread_start_us":634,"threads_started":1}
W20260807 10:49:33.709244 3588 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49252
Aug 07 10:49:33 dist-test-slave-x5fp krb5kdc[3374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 10:49:33 dist-test-slave-x5fp krb5kdc[3374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.193@KRBTEST.COM
I20260807 10:49:33.772912 4129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:33.755292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49260 (local address 127.3.73.193:34633)
0807 10:49:33.755553 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:33.755561 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:33.755810 (+ 249us) server_negotiation.cc:407] Connection header received
0807 10:49:33.756048 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:33.756053 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:33.756128 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:33.756245 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:33.757855 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:33.758735 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:33.760437 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:33.761194 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:33.761305 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:33.766470 (+ 5165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:33.766476 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:33.766480 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:33.766519 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:33.769845 (+ 3326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:33.770665 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:33.770673 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:33.770677 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:33.770839 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:33.771266 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:33.771274 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:33.771278 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:33.771568 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:33.771865 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:33.772222 (+ 357us) server_negotiation.cc:299] Negotiation successful
0807 10:49:33.772496 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260807 10:49:33.775241 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 3529
I20260807 10:49:33.776614 3925 heartbeater.cc:503] Master 127.3.73.254:43411 was elected leader, sending a full tablet report...
I20260807 10:49:33.789561 4129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:33.776556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49268 (local address 127.3.73.193:34633)
0807 10:49:33.776799 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:33.776805 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:33.777106 (+ 301us) server_negotiation.cc:407] Connection header received
0807 10:49:33.777212 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:33.777218 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:33.777357 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:33.777441 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:33.778915 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:33.779645 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:33.781059 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:33.781569 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:33.781685 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:33.784046 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:33.784051 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:33.784054 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:33.784089 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:33.786724 (+ 2635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:33.787386 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:33.787390 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:33.787392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:33.787530 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:33.787906 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:33.787911 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:33.787913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:33.788258 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:33.788475 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:33.788853 (+ 378us) server_negotiation.cc:299] Negotiation successful
0807 10:49:33.789072 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 10:49:33.790397 3710 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:49:33.884389 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 3529
W20260807 10:49:33.913889 3417 connection.cc:535] server connection from 127.3.73.193:34967 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 10:49:33.914312 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 3724
Aug 07 10:49:33 dist-test-slave-x5fp krb5kdc[3374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.194@KRBTEST.COM
I20260807 10:49:33.933773 4146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:33.915626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39810 (local address 127.3.73.194:46075)
0807 10:49:33.916505 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:33.916514 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:33.916545 (+ 31us) server_negotiation.cc:407] Connection header received
0807 10:49:33.916622 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:33.916630 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:33.916961 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:33.917199 (+ 238us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:33.918600 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:33.919748 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:33.921586 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:33.922262 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:33.922445 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:33.927521 (+ 5076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:33.927536 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:33.927553 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:33.927604 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:33.930959 (+ 3355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:33.931732 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:33.931740 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:33.931743 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:33.931884 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:33.932245 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:33.932248 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:33.932250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:33.932629 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:33.932938 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:33.933242 (+ 304us) server_negotiation.cc:299] Negotiation successful
0807 10:49:33.933469 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":706,"thread_start_us":524,"threads_started":1}
I20260807 10:49:33.935220 3919 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:49:34.024567 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 3724
I20260807 10:49:34.049239 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 3931
Aug 07 10:49:34 dist-test-slave-x5fp krb5kdc[3374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.195@KRBTEST.COM
I20260807 10:49:34.067342 4150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:34.050363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48874 (local address 127.3.73.195:34379)
0807 10:49:34.050949 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:34.050957 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:34.050990 (+ 33us) server_negotiation.cc:407] Connection header received
0807 10:49:34.051056 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:34.051063 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:34.051299 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:34.051437 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:34.052898 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:34.054038 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:34.055477 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:34.056159 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:34.056296 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:34.061125 (+ 4829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:34.061139 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:34.061156 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:34.061197 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:34.064207 (+ 3010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:34.065100 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:34.065111 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:34.065115 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:34.065260 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:34.065672 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:34.065675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:34.065678 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:34.066105 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:34.066369 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:34.066739 (+ 370us) server_negotiation.cc:299] Negotiation successful
0807 10:49:34.067019 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":251,"threads_started":1}
I20260807 10:49:34.068671 4106 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:49:34.153512 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 3931
I20260807 10:49:34.179050 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 3396
I20260807 10:49:34.180402 3489 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:49:34.286772 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 3396
2026-08-07T10:49:34Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (6432 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:49:34 dist-test-slave-x5fp krb5kdc[4159](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:49:34 dist-test-slave-x5fp krb5kdc[4159](info): set up 2 sockets
Aug 07 10:49:34 dist-test-slave-x5fp krb5kdc[4159](info): commencing operation
krb5kdc: starting...
W20260807 10:49:35.655236 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.311s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:49:35 dist-test-slave-x5fp krb5kdc[4159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:49:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:49:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.73.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
I20260807 10:49:35.889207 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:40399
--webserver_interface=127.3.73.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:35219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:40399
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:49:36.254158 4177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:49:36.254506 4177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:49:36.254607 4177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:49:36.262379 4177 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:49:36.262516 4177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:49:36.262568 4177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:49:36.262636 4177 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:49:36.262696 4177 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:49:36.273286 4177 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:40399
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:40399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:49:36.274750 4177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:49:36.276190 4177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:49:36.289440 4185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:36.289499 4187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:36.289690 4189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:49:36.290673 4177 server_base.cc:1034] running on GCE node
I20260807 10:49:36.291610 4177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:49:36.293394 4177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:49:36.294656 4177 hybrid_clock.cc:648] HybridClock initialized: now 1786099776294606 us; error 67 us; skew 500 ppm
I20260807 10:49:36.295068 4177 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:49:36 dist-test-slave-x5fp krb5kdc[4159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099776, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:49:36.304409 4177 init.cc:377] Logged in from keytab as kudu/127.3.73.254@KRBTEST.COM (short username kudu)
I20260807 10:49:36.306568 4177 webserver.cc:460] Webserver started at http://127.3.73.254:46387/ using document root <none> and password file <none>
I20260807 10:49:36.307441 4177 fs_manager.cc:359] Metadata directory not provided
I20260807 10:49:36.307626 4177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:49:36.308085 4177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:49:36.311892 4177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "8b7d47800c7c44a7bcbe7eaf33b180f1"
format_stamp: "Formatted at 2026-08-07 10:49:36 on dist-test-slave-x5fp"
server_key: "064989ecb6bf51923705e4a2185d2da3"
server_key_iv: "c90ef2c1d4f0922240d3525ec878d1c6"
server_key_version: "encryptionkey@0"
I20260807 10:49:36.312981 4177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "8b7d47800c7c44a7bcbe7eaf33b180f1"
format_stamp: "Formatted at 2026-08-07 10:49:36 on dist-test-slave-x5fp"
server_key: "064989ecb6bf51923705e4a2185d2da3"
server_key_iv: "c90ef2c1d4f0922240d3525ec878d1c6"
server_key_version: "encryptionkey@0"
I20260807 10:49:36.319406 4177 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 10:49:36.323930 4198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:36.325707 4177 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 10:49:36.325956 4177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "8b7d47800c7c44a7bcbe7eaf33b180f1"
format_stamp: "Formatted at 2026-08-07 10:49:36 on dist-test-slave-x5fp"
server_key: "064989ecb6bf51923705e4a2185d2da3"
server_key_iv: "c90ef2c1d4f0922240d3525ec878d1c6"
server_key_version: "encryptionkey@0"
I20260807 10:49:36.326246 4177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:49:36.352108 4177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:49:36.358004 4177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:49:36.358441 4177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:49:36.382583 4177 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:40399
I20260807 10:49:36.382628 4279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:40399 every 8 connection(s)
I20260807 10:49:36.384436 4177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:49:36.388880 4281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:49:36.394961 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 4177
I20260807 10:49:36.395258 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:49:36.395931 3367 external_mini_cluster.cc:1428] Setting key 2c63a3c69c957bb81d2fce8832770789
I20260807 10:49:36.401679 4281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b7d47800c7c44a7bcbe7eaf33b180f1: Bootstrap starting.
I20260807 10:49:36.406051 4281 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b7d47800c7c44a7bcbe7eaf33b180f1: Neither blocks nor log segments found. Creating new log.
I20260807 10:49:36.407337 4281 log.cc:824] T 00000000000000000000000000000000 P 8b7d47800c7c44a7bcbe7eaf33b180f1: Log is configured to *not* fsync() on all Append() calls
I20260807 10:49:36.410909 4281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b7d47800c7c44a7bcbe7eaf33b180f1: No bootstrap required, opened a new log
Aug 07 10:49:36 dist-test-slave-x5fp krb5kdc[4159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:49:36.421814 4281 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b7d47800c7c44a7bcbe7eaf33b180f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b7d47800c7c44a7bcbe7eaf33b180f1" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 40399 } }
I20260807 10:49:36.422572 4281 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b7d47800c7c44a7bcbe7eaf33b180f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:49:36.422741 4281 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b7d47800c7c44a7bcbe7eaf33b180f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b7d47800c7c44a7bcbe7eaf33b180f1, State: Initialized, Role: FOLLOWER
I20260807 10:49:36.423521 4281 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b7d47800c7c44a7bcbe7eaf33b180f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b7d47800c7c44a7bcbe7eaf33b180f1" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 40399 } }
I20260807 10:49:36.423810 4281 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b7d47800c7c44a7bcbe7eaf33b180f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:49:36.423959 4281 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b7d47800c7c44a7bcbe7eaf33b180f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:49:36.424161 4281 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b7d47800c7c44a7bcbe7eaf33b180f1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:49:36.424341 4285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:36.398351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44870 (local address 127.3.73.254:40399)
0807 10:49:36.399303 (+ 952us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:36.399320 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:36.399375 (+ 55us) server_negotiation.cc:407] Connection header received
0807 10:49:36.401933 (+ 2558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:36.401967 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:36.402513 (+ 546us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:36.402982 (+ 469us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:36.404452 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:36.406625 (+ 2173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:36.408206 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:36.409150 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:36.409389 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:36.414321 (+ 4932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:36.414334 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:36.414360 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:36.414422 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:36.418917 (+ 4495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:36.419801 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:36.419812 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:36.419828 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:36.420110 (+ 282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:36.420525 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:36.420535 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:36.420538 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:36.421364 (+ 826us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:36.421668 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:36.422254 (+ 586us) server_negotiation.cc:299] Negotiation successful
0807 10:49:36.423024 (+ 770us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":408,"threads_started":1}
I20260807 10:49:36.427443 4281 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b7d47800c7c44a7bcbe7eaf33b180f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b7d47800c7c44a7bcbe7eaf33b180f1" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 40399 } }
I20260807 10:49:36.428078 4281 leader_election.cc:302] T 00000000000000000000000000000000 P 8b7d47800c7c44a7bcbe7eaf33b180f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b7d47800c7c44a7bcbe7eaf33b180f1; no voters:
I20260807 10:49:36.428707 4281 leader_election.cc:288] T 00000000000000000000000000000000 P 8b7d47800c7c44a7bcbe7eaf33b180f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:49:36.428958 4287 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b7d47800c7c44a7bcbe7eaf33b180f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:49:36.429710 4287 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b7d47800c7c44a7bcbe7eaf33b180f1 [term 1 LEADER]: Becoming Leader. State: Replica: 8b7d47800c7c44a7bcbe7eaf33b180f1, State: Running, Role: LEADER
I20260807 10:49:36.430368 4287 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b7d47800c7c44a7bcbe7eaf33b180f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b7d47800c7c44a7bcbe7eaf33b180f1" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 40399 } }
I20260807 10:49:36.431087 4281 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b7d47800c7c44a7bcbe7eaf33b180f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:49:36.434211 4288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b7d47800c7c44a7bcbe7eaf33b180f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b7d47800c7c44a7bcbe7eaf33b180f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b7d47800c7c44a7bcbe7eaf33b180f1" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 40399 } } }
I20260807 10:49:36.434947 4288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b7d47800c7c44a7bcbe7eaf33b180f1 [sys.catalog]: This master's current role is: LEADER
I20260807 10:49:36.435649 4289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b7d47800c7c44a7bcbe7eaf33b180f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b7d47800c7c44a7bcbe7eaf33b180f1. Latest consensus state: current_term: 1 leader_uuid: "8b7d47800c7c44a7bcbe7eaf33b180f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b7d47800c7c44a7bcbe7eaf33b180f1" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 40399 } } }
I20260807 10:49:36.436311 4289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b7d47800c7c44a7bcbe7eaf33b180f1 [sys.catalog]: This master's current role is: LEADER
I20260807 10:49:36.437489 4297 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:49:36.443207 4297 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:49:36.453960 4297 catalog_manager.cc:1347] Generated new cluster ID: 6bd8ee777adf49319fd530175c6e8a44
I20260807 10:49:36.454092 4297 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:49:36.473845 4297 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:49:36.475342 4297 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:49:36.484598 4297 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b7d47800c7c44a7bcbe7eaf33b180f1: Generated new TSK 0
I20260807 10:49:36.485576 4297 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.73.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab.
Entry for principal kudu/127.3.73.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab.
I20260807 10:49:36.554658 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.193:0
--local_ip_for_outbound_sockets=127.3.73.193
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:40399
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:35219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:49:36.926674 4314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:49:36.927060 4314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:49:36.927253 4314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:49:36.935081 4314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:49:36.935250 4314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:49:36.935451 4314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.193
I20260807 10:49:36.946599 4314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:40399
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.193
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:49:36.948127 4314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:49:36.949856 4314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:49:36.964596 4321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:36.964638 4327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:36.964552 4324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:49:36.965556 4314 server_base.cc:1034] running on GCE node
I20260807 10:49:36.966338 4314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:49:36.967413 4314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:49:36.968657 4314 hybrid_clock.cc:648] HybridClock initialized: now 1786099776968589 us; error 56 us; skew 500 ppm
I20260807 10:49:36.969107 4314 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:49:36 dist-test-slave-x5fp krb5kdc[4159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099776, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:49:36.972678 4314 init.cc:377] Logged in from keytab as kudu/127.3.73.193@KRBTEST.COM (short username kudu)
I20260807 10:49:36.974527 4314 webserver.cc:460] Webserver started at http://127.3.73.193:32781/ using document root <none> and password file <none>
I20260807 10:49:36.975330 4314 fs_manager.cc:359] Metadata directory not provided
I20260807 10:49:36.975445 4314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:49:36.975791 4314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:49:36.979013 4314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-0/data/instance:
uuid: "3c0f82bda34740a1b9c018904f6b6eee"
format_stamp: "Formatted at 2026-08-07 10:49:36 on dist-test-slave-x5fp"
server_key: "c045faad2933c03235f393ebfca7a69e"
server_key_iv: "2ea305343617a945196b1afa0bf4bbe2"
server_key_version: "encryptionkey@0"
I20260807 10:49:36.979919 4314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance:
uuid: "3c0f82bda34740a1b9c018904f6b6eee"
format_stamp: "Formatted at 2026-08-07 10:49:36 on dist-test-slave-x5fp"
server_key: "c045faad2933c03235f393ebfca7a69e"
server_key_iv: "2ea305343617a945196b1afa0bf4bbe2"
server_key_version: "encryptionkey@0"
I20260807 10:49:36.985963 4314 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 10:49:36.990135 4344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:36.991595 4314 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260807 10:49:36.991793 4314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-0/wal
uuid: "3c0f82bda34740a1b9c018904f6b6eee"
format_stamp: "Formatted at 2026-08-07 10:49:36 on dist-test-slave-x5fp"
server_key: "c045faad2933c03235f393ebfca7a69e"
server_key_iv: "2ea305343617a945196b1afa0bf4bbe2"
server_key_version: "encryptionkey@0"
I20260807 10:49:36.992087 4314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:49:37.013059 4314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:49:37.017879 4314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:49:37.018208 4314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:49:37.019317 4314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:49:37.020874 4314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:49:37.020985 4314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:37.021147 4314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:49:37.021220 4314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:37.061537 4314 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.193:44807
I20260807 10:49:37.061635 4509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.193:44807 every 8 connection(s)
I20260807 10:49:37.062915 4314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
I20260807 10:49:37.064492 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 4314
I20260807 10:49:37.064882 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance
I20260807 10:49:37.065559 3367 external_mini_cluster.cc:1428] Setting key ea6fd0870319ea181fd9b9c1d68d8cb4
Aug 07 10:49:37 dist-test-slave-x5fp krb5kdc[4159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099776, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:49:37.087970 4515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:37.065810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:50527 (local address 127.3.73.254:40399)
0807 10:49:37.066330 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:37.066337 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:37.067362 (+ 1025us) server_negotiation.cc:407] Connection header received
0807 10:49:37.070331 (+ 2969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:37.070336 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:37.070404 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:37.070504 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:37.072862 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:37.073608 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:37.076170 (+ 2562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:37.076730 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:37.076921 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:37.081004 (+ 4083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:37.081019 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:37.081022 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:37.081062 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:37.084034 (+ 2972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:37.085112 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:37.085126 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:37.085129 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:37.085286 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:37.085845 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:37.085850 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:37.085852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:37.086159 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:37.086352 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:37.087241 (+ 889us) server_negotiation.cc:299] Negotiation successful
0807 10:49:37.087514 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":251,"threads_started":1}
I20260807 10:49:37.089183 4516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:37.066409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:40399 (local address 127.3.73.193:50527)
0807 10:49:37.067196 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:37.067245 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:37.069964 (+ 2719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:37.070820 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:37.070833 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:37.071663 (+ 830us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:49:37.072597 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:37.072629 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:37.073784 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:37.073793 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:37.075973 (+ 2180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:37.075993 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:37.077001 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:37.077006 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:37.077217 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:37.078010 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:49:37.078041 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:49:37.080625 (+ 2584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:49:37.084269 (+ 3644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:37.084280 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:37.084301 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:37.084950 (+ 649us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:37.085427 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:37.085433 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:37.085437 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:37.085690 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:37.086328 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:49:37.086338 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:49:37.086774 (+ 436us) client_negotiation.cc:769] Sending connection context
0807 10:49:37.087113 (+ 339us) client_negotiation.cc:240] Negotiation successful
0807 10:49:37.087750 (+ 637us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":359,"threads_started":1}
WARNING: no policy specified for kudu/127.3.73.194@KRBTEST.COM; defaulting to no policy
I20260807 10:49:37.091199 4513 heartbeater.cc:347] Connected to a master server at 127.3.73.254:40399
I20260807 10:49:37.091565 4513 heartbeater.cc:464] Registering TS with master...
I20260807 10:49:37.092465 4513 heartbeater.cc:511] Master 127.3.73.254:40399 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.194@KRBTEST.COM" created.
I20260807 10:49:37.095041 4222 ts_manager.cc:194] Registered new tserver with Master: 3c0f82bda34740a1b9c018904f6b6eee (127.3.73.193:44807)
I20260807 10:49:37.096810 4222 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.193@KRBTEST.COM'} at 127.3.73.193:50527
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab.
Entry for principal kudu/127.3.73.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab.
I20260807 10:49:37.124320 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.194:0
--local_ip_for_outbound_sockets=127.3.73.194
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:40399
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:35219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:49:37.486768 4521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:49:37.487100 4521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:49:37.487279 4521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:49:37.494726 4521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:49:37.494899 4521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:49:37.495082 4521 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.194
I20260807 10:49:37.506052 4521 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:40399
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.194
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:49:37.507702 4521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:49:37.509223 4521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:49:37.526648 4531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:37.528036 4534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:37.528038 4530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:49:37.528486 4521 server_base.cc:1034] running on GCE node
I20260807 10:49:37.529318 4521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:49:37.530382 4521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:49:37.531718 4521 hybrid_clock.cc:648] HybridClock initialized: now 1786099777531577 us; error 57 us; skew 500 ppm
I20260807 10:49:37.532148 4521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:49:37 dist-test-slave-x5fp krb5kdc[4159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099777, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:49:37.536149 4521 init.cc:377] Logged in from keytab as kudu/127.3.73.194@KRBTEST.COM (short username kudu)
I20260807 10:49:37.538009 4521 webserver.cc:460] Webserver started at http://127.3.73.194:33065/ using document root <none> and password file <none>
I20260807 10:49:37.538825 4521 fs_manager.cc:359] Metadata directory not provided
I20260807 10:49:37.538985 4521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:49:37.539345 4521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:49:37.543141 4521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-1/data/instance:
uuid: "1def62b5dd654258b4249815c6b149bc"
format_stamp: "Formatted at 2026-08-07 10:49:37 on dist-test-slave-x5fp"
server_key: "d824e4c1aee0dc2f2f9852cd3a578635"
server_key_iv: "42524ff4e99e2b8d7b8d73ff752e04bc"
server_key_version: "encryptionkey@0"
I20260807 10:49:37.544461 4521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance:
uuid: "1def62b5dd654258b4249815c6b149bc"
format_stamp: "Formatted at 2026-08-07 10:49:37 on dist-test-slave-x5fp"
server_key: "d824e4c1aee0dc2f2f9852cd3a578635"
server_key_iv: "42524ff4e99e2b8d7b8d73ff752e04bc"
server_key_version: "encryptionkey@0"
I20260807 10:49:37.550693 4521 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260807 10:49:37.555086 4541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:37.557191 4521 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 10:49:37.557440 4521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-1/wal
uuid: "1def62b5dd654258b4249815c6b149bc"
format_stamp: "Formatted at 2026-08-07 10:49:37 on dist-test-slave-x5fp"
server_key: "d824e4c1aee0dc2f2f9852cd3a578635"
server_key_iv: "42524ff4e99e2b8d7b8d73ff752e04bc"
server_key_version: "encryptionkey@0"
I20260807 10:49:37.557686 4521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:49:37.578068 4521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:49:37.582964 4521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:49:37.583288 4521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:49:37.584383 4521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:49:37.585975 4521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:49:37.586081 4521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:37.586236 4521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:49:37.586310 4521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:37.628371 4521 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.194:46031
I20260807 10:49:37.628403 4709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.194:46031 every 8 connection(s)
I20260807 10:49:37.629935 4521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
I20260807 10:49:37.630412 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 4521
I20260807 10:49:37.630633 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance
I20260807 10:49:37.631238 3367 external_mini_cluster.cc:1428] Setting key f20eceeb84caf60505b278e7107dac1f
Aug 07 10:49:37 dist-test-slave-x5fp krb5kdc[4159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099777, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:49:37.654989 4714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:37.633094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:33453 (local address 127.3.73.254:40399)
0807 10:49:37.633597 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:37.633604 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:37.634778 (+ 1174us) server_negotiation.cc:407] Connection header received
0807 10:49:37.636959 (+ 2181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:37.636966 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:37.637062 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:37.637208 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:37.639030 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:37.639866 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:37.641667 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:37.642161 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:37.642323 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:37.647273 (+ 4950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:37.647282 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:37.647289 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:37.647337 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:37.651216 (+ 3879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:37.652086 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:37.652095 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:37.652100 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:37.652271 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:37.652814 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:37.652822 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:37.652828 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:37.653257 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:37.653445 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:37.654198 (+ 753us) server_negotiation.cc:299] Negotiation successful
0807 10:49:37.654460 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":245,"threads_started":1}
I20260807 10:49:37.655758 4715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:37.633749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:40399 (local address 127.3.73.194:33453)
0807 10:49:37.634605 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:37.634648 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:37.636596 (+ 1948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:37.637553 (+ 957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:37.637563 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:37.638139 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:49:37.638841 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:37.638861 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:37.640044 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:37.640050 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:37.641492 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:37.641505 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:37.642439 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:37.642448 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:37.642708 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:37.643584 (+ 876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:49:37.643631 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:49:37.646997 (+ 3366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:49:37.651400 (+ 4403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:37.651408 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:37.651421 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:37.651919 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:37.652429 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:37.652433 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:37.652435 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:37.652647 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:37.653403 (+ 756us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:49:37.653409 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:49:37.653772 (+ 363us) client_negotiation.cc:769] Sending connection context
0807 10:49:37.654024 (+ 252us) client_negotiation.cc:240] Negotiation successful
0807 10:49:37.654540 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":344,"threads_started":1}
I20260807 10:49:37.657380 4712 heartbeater.cc:347] Connected to a master server at 127.3.73.254:40399
I20260807 10:49:37.657786 4712 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.3.73.195@KRBTEST.COM; defaulting to no policy
I20260807 10:49:37.658779 4712 heartbeater.cc:511] Master 127.3.73.254:40399 requested a full tablet report, sending...
I20260807 10:49:37.660570 4222 ts_manager.cc:194] Registered new tserver with Master: 1def62b5dd654258b4249815c6b149bc (127.3.73.194:46031)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.195@KRBTEST.COM" created.
I20260807 10:49:37.661610 4222 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.194@KRBTEST.COM'} at 127.3.73.194:33453
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab.
Entry for principal kudu/127.3.73.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab.
I20260807 10:49:37.692611 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.195:0
--local_ip_for_outbound_sockets=127.3.73.195
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:40399
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:35219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:49:38.052325 4719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:49:38.052677 4719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:49:38.052906 4719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:49:38.060532 4719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:49:38.060667 4719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:49:38.060940 4719 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.195
I20260807 10:49:38.071532 4719 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:40399
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.195
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:49:38.073256 4719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:49:38.074920 4719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:49:38.091706 4728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:38.091732 4732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:38.092042 4726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:49:38.092785 4719 server_base.cc:1034] running on GCE node
I20260807 10:49:38.093528 4719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:49:38.094589 4719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:49:38.095919 4719 hybrid_clock.cc:648] HybridClock initialized: now 1786099778095836 us; error 68 us; skew 500 ppm
I20260807 10:49:38.096308 4719 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:49:38 dist-test-slave-x5fp krb5kdc[4159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099778, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:49:38.100119 4513 heartbeater.cc:503] Master 127.3.73.254:40399 was elected leader, sending a full tablet report...
I20260807 10:49:38.100119 4719 init.cc:377] Logged in from keytab as kudu/127.3.73.195@KRBTEST.COM (short username kudu)
I20260807 10:49:38.102455 4719 webserver.cc:460] Webserver started at http://127.3.73.195:44839/ using document root <none> and password file <none>
I20260807 10:49:38.103247 4719 fs_manager.cc:359] Metadata directory not provided
I20260807 10:49:38.103367 4719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:49:38.103668 4719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:49:38.106985 4719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-2/data/instance:
uuid: "44fd8ee8288a41f49b575ab308cd968d"
format_stamp: "Formatted at 2026-08-07 10:49:38 on dist-test-slave-x5fp"
server_key: "7a6e3ce48600b5684e86c07289365b09"
server_key_iv: "8222a5f3d9541810d4fea9dc795aa307"
server_key_version: "encryptionkey@0"
I20260807 10:49:38.107816 4719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance:
uuid: "44fd8ee8288a41f49b575ab308cd968d"
format_stamp: "Formatted at 2026-08-07 10:49:38 on dist-test-slave-x5fp"
server_key: "7a6e3ce48600b5684e86c07289365b09"
server_key_iv: "8222a5f3d9541810d4fea9dc795aa307"
server_key_version: "encryptionkey@0"
I20260807 10:49:38.113570 4719 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 10:49:38.117664 4742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:38.119506 4719 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 10:49:38.119760 4719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-2/wal
uuid: "44fd8ee8288a41f49b575ab308cd968d"
format_stamp: "Formatted at 2026-08-07 10:49:38 on dist-test-slave-x5fp"
server_key: "7a6e3ce48600b5684e86c07289365b09"
server_key_iv: "8222a5f3d9541810d4fea9dc795aa307"
server_key_version: "encryptionkey@0"
I20260807 10:49:38.120016 4719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:49:38.157404 4719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:49:38.162573 4719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:49:38.162981 4719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:49:38.164194 4719 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:49:38.166066 4719 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:49:38.166154 4719 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:38.166343 4719 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:49:38.166402 4719 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:38.223222 4719 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.195:40471
I20260807 10:49:38.223228 4924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.195:40471 every 8 connection(s)
I20260807 10:49:38.225112 4719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
I20260807 10:49:38.231052 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 4719
I20260807 10:49:38.231519 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance
I20260807 10:49:38.232247 3367 external_mini_cluster.cc:1428] Setting key 504416ceac2a9f4264acea58a31c7123
Aug 07 10:49:38 dist-test-slave-x5fp krb5kdc[4159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099778, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:49:38.257252 4928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:38.229571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:41021 (local address 127.3.73.254:40399)
0807 10:49:38.230869 (+ 1298us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:38.230877 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:38.232480 (+ 1603us) server_negotiation.cc:407] Connection header received
0807 10:49:38.236638 (+ 4158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:38.236643 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:38.236790 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:38.236938 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:38.239087 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:38.240351 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:38.242535 (+ 2184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:38.243185 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:38.243331 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:38.248771 (+ 5440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:38.248781 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:38.248789 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:38.248840 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:38.253235 (+ 4395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:38.254146 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:38.254155 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:38.254161 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:38.254330 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:38.254897 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:38.254906 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:38.254912 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:38.255321 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:38.255500 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:38.256304 (+ 804us) server_negotiation.cc:299] Negotiation successful
0807 10:49:38.256583 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1091,"thread_start_us":753,"threads_started":1}
I20260807 10:49:38.258167 4931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:38.230158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:40399 (local address 127.3.73.195:41021)
0807 10:49:38.231883 (+ 1725us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:38.231926 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:38.236297 (+ 4371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:38.237287 (+ 990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:38.237296 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:38.238040 (+ 744us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:49:38.238903 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:38.238927 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:38.240555 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:38.240559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:38.242378 (+ 1819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:38.242392 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:38.243585 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:38.243597 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:38.243964 (+ 367us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:38.244855 (+ 891us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:49:38.244895 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:49:38.248462 (+ 3567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:49:38.253417 (+ 4955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:38.253425 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:38.253439 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:38.254013 (+ 574us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:38.254453 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:38.254456 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:38.254459 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:38.254737 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:38.255457 (+ 720us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:49:38.255463 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:49:38.255895 (+ 432us) client_negotiation.cc:769] Sending connection context
0807 10:49:38.256143 (+ 248us) client_negotiation.cc:240] Negotiation successful
0807 10:49:38.256629 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1303,"thread_start_us":936,"threads_started":1}
I20260807 10:49:38.260406 4925 heartbeater.cc:347] Connected to a master server at 127.3.73.254:40399
I20260807 10:49:38.260885 4925 heartbeater.cc:464] Registering TS with master...
I20260807 10:49:38.262281 4925 heartbeater.cc:511] Master 127.3.73.254:40399 requested a full tablet report, sending...
I20260807 10:49:38.264403 4222 ts_manager.cc:194] Registered new tserver with Master: 44fd8ee8288a41f49b575ab308cd968d (127.3.73.195:40471)
I20260807 10:49:38.265455 4222 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.195@KRBTEST.COM'} at 127.3.73.195:41021
I20260807 10:49:38.274766 3367 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 10:49:38.301258 4928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:38.284614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44878 (local address 127.3.73.254:40399)
0807 10:49:38.284855 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:38.284860 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:38.285428 (+ 568us) server_negotiation.cc:407] Connection header received
0807 10:49:38.285577 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:38.285581 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:38.285654 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:38.285763 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:38.287700 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:38.288850 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:38.290816 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:38.291473 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:38.291580 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:38.294928 (+ 3348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:38.294934 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:38.294937 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:38.294982 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:38.298043 (+ 3061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:38.298900 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:38.298904 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:38.298906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:38.299044 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:38.299514 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:38.299517 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:38.299519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:38.299869 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:38.300066 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:38.300614 (+ 548us) server_negotiation.cc:299] Negotiation successful
0807 10:49:38.300895 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 10:49:38.311174 4222 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44878:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 10:49:38.321489 4222 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 10:49:38.358191 4943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:38.341439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49298 (local address 127.3.73.195:40471)
0807 10:49:38.342745 (+ 1306us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:38.342757 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:38.342831 (+ 74us) server_negotiation.cc:407] Connection header received
0807 10:49:38.342966 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:38.342974 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:38.343318 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:38.343583 (+ 265us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:49:38.345296 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:38.347493 (+ 2197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:38.353194 (+ 5701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:38.355737 (+ 2543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:38.355866 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:38.356224 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:38.357184 (+ 960us) server_negotiation.cc:299] Negotiation successful
0807 10:49:38.357503 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1024,"thread_start_us":712,"threads_started":1}
I20260807 10:49:38.359858 4944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:38.341590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:40471 (local address 127.0.0.1:49298)
0807 10:49:38.342512 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:38.342546 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:38.342720 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:38.343592 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:38.343599 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:38.343631 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:49:38.344978 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:38.344999 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:38.347685 (+ 2686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:38.347695 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:38.352937 (+ 5242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:38.352955 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:38.356482 (+ 3527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:38.356490 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:38.356873 (+ 383us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:38.356938 (+ 65us) client_negotiation.cc:769] Sending connection context
0807 10:49:38.358556 (+ 1618us) client_negotiation.cc:240] Negotiation successful
0807 10:49:38.358826 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"mutex_wait_us":21,"thread_start_us":547,"threads_started":1}
I20260807 10:49:38.360914 4946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:38.343584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42082 (local address 127.3.73.194:46031)
0807 10:49:38.349075 (+ 5491us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:38.349084 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:38.349116 (+ 32us) server_negotiation.cc:407] Connection header received
0807 10:49:38.349231 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:38.349240 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:38.349503 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:38.349719 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:49:38.350869 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:38.352906 (+ 2037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:38.356977 (+ 4071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:38.358759 (+ 1782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:38.359117 (+ 358us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:38.359353 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:38.359920 (+ 567us) server_negotiation.cc:299] Negotiation successful
0807 10:49:38.360169 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5342,"thread_start_us":5163,"threads_started":1}
I20260807 10:49:38.361202 4945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:38.343289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:46031 (local address 127.0.0.1:42082)
0807 10:49:38.348366 (+ 5077us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:38.348391 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:38.348531 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:38.349696 (+ 1165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:38.349700 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:38.349727 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:49:38.350440 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:38.350456 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:38.353085 (+ 2629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:38.353090 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:38.356717 (+ 3627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:38.356762 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:38.359263 (+ 2501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:38.359276 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:38.359557 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:38.359650 (+ 93us) client_negotiation.cc:769] Sending connection context
0807 10:49:38.359808 (+ 158us) client_negotiation.cc:240] Negotiation successful
0807 10:49:38.360237 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4900,"mutex_wait_us":28,"thread_start_us":4681,"threads_started":1}
I20260807 10:49:38.366217 4948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:38.350081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:44807 (local address 127.0.0.1:37578)
0807 10:49:38.351829 (+ 1748us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:38.351847 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:38.352097 (+ 250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:38.353483 (+ 1386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:38.353489 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:38.353514 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:49:38.354837 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:38.354850 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:38.357125 (+ 2275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:38.357130 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:38.362327 (+ 5197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:38.362345 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:38.364918 (+ 2573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:38.364931 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:38.365207 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:38.365243 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 10:49:38.365347 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 10:49:38.365487 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1614,"thread_start_us":1988,"threads_started":1}
I20260807 10:49:38.365949 4616 tablet_service.cc:1467] Processing CreateTablet for tablet 10b673962a024d6abc7ef41c9cbae32e (DEFAULT_TABLE table=test-table [id=b09c9132aafb44d08e3e7f2ea0e5a66b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:49:38.367555 4947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:38.349272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37578 (local address 127.3.73.193:44807)
0807 10:49:38.351156 (+ 1884us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:38.351169 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:38.351965 (+ 796us) server_negotiation.cc:407] Connection header received
0807 10:49:38.352316 (+ 351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:38.352323 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:38.352619 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:38.352898 (+ 279us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:49:38.355103 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:38.356952 (+ 1849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:38.362578 (+ 5626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:38.364560 (+ 1982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:38.364972 (+ 412us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:38.365211 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:38.365456 (+ 245us) server_negotiation.cc:299] Negotiation successful
0807 10:49:38.367016 (+ 1560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1612,"thread_start_us":1337,"threads_started":1}
I20260807 10:49:38.367962 4616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10b673962a024d6abc7ef41c9cbae32e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:49:38.368683 4816 tablet_service.cc:1467] Processing CreateTablet for tablet 10b673962a024d6abc7ef41c9cbae32e (DEFAULT_TABLE table=test-table [id=b09c9132aafb44d08e3e7f2ea0e5a66b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:49:38.370445 4816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10b673962a024d6abc7ef41c9cbae32e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:49:38.372442 4414 tablet_service.cc:1467] Processing CreateTablet for tablet 10b673962a024d6abc7ef41c9cbae32e (DEFAULT_TABLE table=test-table [id=b09c9132aafb44d08e3e7f2ea0e5a66b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:49:38.374809 4414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10b673962a024d6abc7ef41c9cbae32e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:49:38.406360 4949 tablet_bootstrap.cc:492] T 10b673962a024d6abc7ef41c9cbae32e P 1def62b5dd654258b4249815c6b149bc: Bootstrap starting.
I20260807 10:49:38.408536 4951 tablet_bootstrap.cc:492] T 10b673962a024d6abc7ef41c9cbae32e P 44fd8ee8288a41f49b575ab308cd968d: Bootstrap starting.
I20260807 10:49:38.410730 4952 tablet_bootstrap.cc:492] T 10b673962a024d6abc7ef41c9cbae32e P 3c0f82bda34740a1b9c018904f6b6eee: Bootstrap starting.
I20260807 10:49:38.414131 4951 tablet_bootstrap.cc:654] T 10b673962a024d6abc7ef41c9cbae32e P 44fd8ee8288a41f49b575ab308cd968d: Neither blocks nor log segments found. Creating new log.
I20260807 10:49:38.416395 4951 log.cc:824] T 10b673962a024d6abc7ef41c9cbae32e P 44fd8ee8288a41f49b575ab308cd968d: Log is configured to *not* fsync() on all Append() calls
I20260807 10:49:38.417323 4949 tablet_bootstrap.cc:654] T 10b673962a024d6abc7ef41c9cbae32e P 1def62b5dd654258b4249815c6b149bc: Neither blocks nor log segments found. Creating new log.
I20260807 10:49:38.417301 4952 tablet_bootstrap.cc:654] T 10b673962a024d6abc7ef41c9cbae32e P 3c0f82bda34740a1b9c018904f6b6eee: Neither blocks nor log segments found. Creating new log.
I20260807 10:49:38.419662 4949 log.cc:824] T 10b673962a024d6abc7ef41c9cbae32e P 1def62b5dd654258b4249815c6b149bc: Log is configured to *not* fsync() on all Append() calls
I20260807 10:49:38.419639 4952 log.cc:824] T 10b673962a024d6abc7ef41c9cbae32e P 3c0f82bda34740a1b9c018904f6b6eee: Log is configured to *not* fsync() on all Append() calls
I20260807 10:49:38.427832 4951 tablet_bootstrap.cc:492] T 10b673962a024d6abc7ef41c9cbae32e P 44fd8ee8288a41f49b575ab308cd968d: No bootstrap required, opened a new log
I20260807 10:49:38.428478 4951 ts_tablet_manager.cc:1397] T 10b673962a024d6abc7ef41c9cbae32e P 44fd8ee8288a41f49b575ab308cd968d: Time spent bootstrapping tablet: real 0.020s user 0.008s sys 0.009s
I20260807 10:49:38.430824 4949 tablet_bootstrap.cc:492] T 10b673962a024d6abc7ef41c9cbae32e P 1def62b5dd654258b4249815c6b149bc: No bootstrap required, opened a new log
I20260807 10:49:38.431366 4949 ts_tablet_manager.cc:1397] T 10b673962a024d6abc7ef41c9cbae32e P 1def62b5dd654258b4249815c6b149bc: Time spent bootstrapping tablet: real 0.025s user 0.009s sys 0.008s
I20260807 10:49:38.431699 4952 tablet_bootstrap.cc:492] T 10b673962a024d6abc7ef41c9cbae32e P 3c0f82bda34740a1b9c018904f6b6eee: No bootstrap required, opened a new log
I20260807 10:49:38.432379 4952 ts_tablet_manager.cc:1397] T 10b673962a024d6abc7ef41c9cbae32e P 3c0f82bda34740a1b9c018904f6b6eee: Time spent bootstrapping tablet: real 0.022s user 0.009s sys 0.010s
I20260807 10:49:38.441520 4951 raft_consensus.cc:348] T 10b673962a024d6abc7ef41c9cbae32e P 44fd8ee8288a41f49b575ab308cd968d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fd8ee8288a41f49b575ab308cd968d" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 40471 } } peers { permanent_uuid: "1def62b5dd654258b4249815c6b149bc" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46031 } } peers { permanent_uuid: "3c0f82bda34740a1b9c018904f6b6eee" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44807 } }
I20260807 10:49:38.442276 4951 raft_consensus.cc:374] T 10b673962a024d6abc7ef41c9cbae32e P 44fd8ee8288a41f49b575ab308cd968d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:49:38.442497 4951 raft_consensus.cc:729] T 10b673962a024d6abc7ef41c9cbae32e P 44fd8ee8288a41f49b575ab308cd968d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44fd8ee8288a41f49b575ab308cd968d, State: Initialized, Role: FOLLOWER
I20260807 10:49:38.443367 4951 consensus_queue.cc:260] T 10b673962a024d6abc7ef41c9cbae32e P 44fd8ee8288a41f49b575ab308cd968d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fd8ee8288a41f49b575ab308cd968d" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 40471 } } peers { permanent_uuid: "1def62b5dd654258b4249815c6b149bc" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46031 } } peers { permanent_uuid: "3c0f82bda34740a1b9c018904f6b6eee" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44807 } }
I20260807 10:49:38.445525 4925 heartbeater.cc:503] Master 127.3.73.254:40399 was elected leader, sending a full tablet report...
I20260807 10:49:38.446455 4951 ts_tablet_manager.cc:1428] T 10b673962a024d6abc7ef41c9cbae32e P 44fd8ee8288a41f49b575ab308cd968d: Time spent starting tablet: real 0.018s user 0.011s sys 0.007s
I20260807 10:49:38.451813 4949 raft_consensus.cc:348] T 10b673962a024d6abc7ef41c9cbae32e P 1def62b5dd654258b4249815c6b149bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fd8ee8288a41f49b575ab308cd968d" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 40471 } } peers { permanent_uuid: "1def62b5dd654258b4249815c6b149bc" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46031 } } peers { permanent_uuid: "3c0f82bda34740a1b9c018904f6b6eee" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44807 } }
I20260807 10:49:38.451800 4952 raft_consensus.cc:348] T 10b673962a024d6abc7ef41c9cbae32e P 3c0f82bda34740a1b9c018904f6b6eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fd8ee8288a41f49b575ab308cd968d" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 40471 } } peers { permanent_uuid: "1def62b5dd654258b4249815c6b149bc" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46031 } } peers { permanent_uuid: "3c0f82bda34740a1b9c018904f6b6eee" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44807 } }
I20260807 10:49:38.452497 4952 raft_consensus.cc:374] T 10b673962a024d6abc7ef41c9cbae32e P 3c0f82bda34740a1b9c018904f6b6eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:49:38.452515 4949 raft_consensus.cc:374] T 10b673962a024d6abc7ef41c9cbae32e P 1def62b5dd654258b4249815c6b149bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:49:38.452759 4949 raft_consensus.cc:729] T 10b673962a024d6abc7ef41c9cbae32e P 1def62b5dd654258b4249815c6b149bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1def62b5dd654258b4249815c6b149bc, State: Initialized, Role: FOLLOWER
I20260807 10:49:38.452759 4952 raft_consensus.cc:729] T 10b673962a024d6abc7ef41c9cbae32e P 3c0f82bda34740a1b9c018904f6b6eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c0f82bda34740a1b9c018904f6b6eee, State: Initialized, Role: FOLLOWER
I20260807 10:49:38.453670 4952 consensus_queue.cc:260] T 10b673962a024d6abc7ef41c9cbae32e P 3c0f82bda34740a1b9c018904f6b6eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fd8ee8288a41f49b575ab308cd968d" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 40471 } } peers { permanent_uuid: "1def62b5dd654258b4249815c6b149bc" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46031 } } peers { permanent_uuid: "3c0f82bda34740a1b9c018904f6b6eee" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44807 } }
I20260807 10:49:38.453616 4949 consensus_queue.cc:260] T 10b673962a024d6abc7ef41c9cbae32e P 1def62b5dd654258b4249815c6b149bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fd8ee8288a41f49b575ab308cd968d" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 40471 } } peers { permanent_uuid: "1def62b5dd654258b4249815c6b149bc" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46031 } } peers { permanent_uuid: "3c0f82bda34740a1b9c018904f6b6eee" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44807 } }
I20260807 10:49:38.455607 4712 heartbeater.cc:503] Master 127.3.73.254:40399 was elected leader, sending a full tablet report...
I20260807 10:49:38.456563 4949 ts_tablet_manager.cc:1428] T 10b673962a024d6abc7ef41c9cbae32e P 1def62b5dd654258b4249815c6b149bc: Time spent starting tablet: real 0.025s user 0.016s sys 0.007s
I20260807 10:49:38.457792 4952 ts_tablet_manager.cc:1428] T 10b673962a024d6abc7ef41c9cbae32e P 3c0f82bda34740a1b9c018904f6b6eee: Time spent starting tablet: real 0.025s user 0.012s sys 0.013s
W20260807 10:49:38.478686 4927 tablet.cc:2369] T 10b673962a024d6abc7ef41c9cbae32e P 44fd8ee8288a41f49b575ab308cd968d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 10:49:38.566056 4514 tablet.cc:2369] T 10b673962a024d6abc7ef41c9cbae32e P 3c0f82bda34740a1b9c018904f6b6eee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:49:38.574633 4963 raft_consensus.cc:482] T 10b673962a024d6abc7ef41c9cbae32e P 3c0f82bda34740a1b9c018904f6b6eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 10:49:38.575184 4963 raft_consensus.cc:504] T 10b673962a024d6abc7ef41c9cbae32e P 3c0f82bda34740a1b9c018904f6b6eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fd8ee8288a41f49b575ab308cd968d" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 40471 } } peers { permanent_uuid: "1def62b5dd654258b4249815c6b149bc" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46031 } } peers { permanent_uuid: "3c0f82bda34740a1b9c018904f6b6eee" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44807 } }
I20260807 10:49:38.578533 4963 leader_election.cc:288] T 10b673962a024d6abc7ef41c9cbae32e P 3c0f82bda34740a1b9c018904f6b6eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 44fd8ee8288a41f49b575ab308cd968d (127.3.73.195:40471), 1def62b5dd654258b4249815c6b149bc (127.3.73.194:46031)
I20260807 10:49:38.589432 4943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:38.579035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:40263 (local address 127.3.73.195:40471)
0807 10:49:38.579297 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:38.579302 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:38.580198 (+ 896us) server_negotiation.cc:407] Connection header received
0807 10:49:38.580642 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:38.580652 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:38.580783 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:38.580887 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:49:38.582568 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:38.583607 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:38.586551 (+ 2944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:38.588165 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:38.588484 (+ 319us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:38.588575 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:38.588886 (+ 311us) server_negotiation.cc:299] Negotiation successful
0807 10:49:38.589032 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 10:49:38.589596 4964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:38.578854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:40471 (local address 127.3.73.193:40263)
0807 10:49:38.580060 (+ 1206us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:38.580084 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:38.580311 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:38.581282 (+ 971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:38.581290 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:38.581325 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:49:38.582398 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:38.582411 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:38.583765 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:38.583774 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:38.586346 (+ 2572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:38.586361 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:38.588434 (+ 2073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:38.588443 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:38.588665 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:38.588692 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 10:49:38.588864 (+ 172us) client_negotiation.cc:240] Negotiation successful
0807 10:49:38.589018 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1045,"mutex_wait_us":29,"thread_start_us":666,"threads_started":1}
I20260807 10:49:38.591311 4946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:38.581164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:52587 (local address 127.3.73.194:46031)
0807 10:49:38.581554 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:38.581562 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:38.582148 (+ 586us) server_negotiation.cc:407] Connection header received
0807 10:49:38.582434 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:38.582441 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:38.582511 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:38.582621 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:49:38.583907 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:38.584956 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:38.588004 (+ 3048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:38.589441 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:38.589649 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:38.589736 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:38.590745 (+ 1009us) server_negotiation.cc:299] Negotiation successful
0807 10:49:38.591007 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260807 10:49:38.591645 4965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:38.580253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:46031 (local address 127.3.73.193:52587)
0807 10:49:38.582026 (+ 1773us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:38.582043 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:38.582260 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:38.582640 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:38.582645 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:38.582670 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:49:38.583693 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:38.583706 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:38.585125 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:38.585131 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:38.587786 (+ 2655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:38.587801 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:38.590201 (+ 2400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:38.590207 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:38.590514 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:38.590563 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 10:49:38.590724 (+ 161us) client_negotiation.cc:240] Negotiation successful
0807 10:49:38.590973 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1594,"mutex_wait_us":1,"thread_start_us":1013,"threads_started":1}
I20260807 10:49:38.591542 4855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10b673962a024d6abc7ef41c9cbae32e" candidate_uuid: "3c0f82bda34740a1b9c018904f6b6eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44fd8ee8288a41f49b575ab308cd968d" is_pre_election: true
I20260807 10:49:38.592494 4855 raft_consensus.cc:2454] T 10b673962a024d6abc7ef41c9cbae32e P 44fd8ee8288a41f49b575ab308cd968d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c0f82bda34740a1b9c018904f6b6eee in term 0.
I20260807 10:49:38.593252 4645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10b673962a024d6abc7ef41c9cbae32e" candidate_uuid: "3c0f82bda34740a1b9c018904f6b6eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1def62b5dd654258b4249815c6b149bc" is_pre_election: true
I20260807 10:49:38.593829 4347 leader_election.cc:302] T 10b673962a024d6abc7ef41c9cbae32e P 3c0f82bda34740a1b9c018904f6b6eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c0f82bda34740a1b9c018904f6b6eee, 44fd8ee8288a41f49b575ab308cd968d; no voters:
I20260807 10:49:38.593897 4645 raft_consensus.cc:2454] T 10b673962a024d6abc7ef41c9cbae32e P 1def62b5dd654258b4249815c6b149bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c0f82bda34740a1b9c018904f6b6eee in term 0.
I20260807 10:49:38.594463 4963 raft_consensus.cc:2781] T 10b673962a024d6abc7ef41c9cbae32e P 3c0f82bda34740a1b9c018904f6b6eee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 10:49:38.594748 4963 raft_consensus.cc:482] T 10b673962a024d6abc7ef41c9cbae32e P 3c0f82bda34740a1b9c018904f6b6eee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 10:49:38.594942 4963 raft_consensus.cc:3037] T 10b673962a024d6abc7ef41c9cbae32e P 3c0f82bda34740a1b9c018904f6b6eee [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:49:38.598529 4963 raft_consensus.cc:504] T 10b673962a024d6abc7ef41c9cbae32e P 3c0f82bda34740a1b9c018904f6b6eee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fd8ee8288a41f49b575ab308cd968d" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 40471 } } peers { permanent_uuid: "1def62b5dd654258b4249815c6b149bc" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46031 } } peers { permanent_uuid: "3c0f82bda34740a1b9c018904f6b6eee" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44807 } }
I20260807 10:49:38.599540 4963 leader_election.cc:288] T 10b673962a024d6abc7ef41c9cbae32e P 3c0f82bda34740a1b9c018904f6b6eee [CANDIDATE]: Term 1 election: Requested vote from peers 44fd8ee8288a41f49b575ab308cd968d (127.3.73.195:40471), 1def62b5dd654258b4249815c6b149bc (127.3.73.194:46031)
I20260807 10:49:38.600078 4855 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10b673962a024d6abc7ef41c9cbae32e" candidate_uuid: "3c0f82bda34740a1b9c018904f6b6eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44fd8ee8288a41f49b575ab308cd968d"
I20260807 10:49:38.600272 4645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10b673962a024d6abc7ef41c9cbae32e" candidate_uuid: "3c0f82bda34740a1b9c018904f6b6eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1def62b5dd654258b4249815c6b149bc"
I20260807 10:49:38.600330 4855 raft_consensus.cc:3037] T 10b673962a024d6abc7ef41c9cbae32e P 44fd8ee8288a41f49b575ab308cd968d [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:49:38.600555 4645 raft_consensus.cc:3037] T 10b673962a024d6abc7ef41c9cbae32e P 1def62b5dd654258b4249815c6b149bc [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:49:38.603844 4855 raft_consensus.cc:2454] T 10b673962a024d6abc7ef41c9cbae32e P 44fd8ee8288a41f49b575ab308cd968d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c0f82bda34740a1b9c018904f6b6eee in term 1.
I20260807 10:49:38.604013 4645 raft_consensus.cc:2454] T 10b673962a024d6abc7ef41c9cbae32e P 1def62b5dd654258b4249815c6b149bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c0f82bda34740a1b9c018904f6b6eee in term 1.
I20260807 10:49:38.604552 4347 leader_election.cc:302] T 10b673962a024d6abc7ef41c9cbae32e P 3c0f82bda34740a1b9c018904f6b6eee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c0f82bda34740a1b9c018904f6b6eee, 44fd8ee8288a41f49b575ab308cd968d; no voters:
I20260807 10:49:38.605077 4963 raft_consensus.cc:2781] T 10b673962a024d6abc7ef41c9cbae32e P 3c0f82bda34740a1b9c018904f6b6eee [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:49:38.606029 4963 raft_consensus.cc:686] T 10b673962a024d6abc7ef41c9cbae32e P 3c0f82bda34740a1b9c018904f6b6eee [term 1 LEADER]: Becoming Leader. State: Replica: 3c0f82bda34740a1b9c018904f6b6eee, State: Running, Role: LEADER
I20260807 10:49:38.606710 4963 consensus_queue.cc:237] T 10b673962a024d6abc7ef41c9cbae32e P 3c0f82bda34740a1b9c018904f6b6eee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fd8ee8288a41f49b575ab308cd968d" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 40471 } } peers { permanent_uuid: "1def62b5dd654258b4249815c6b149bc" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46031 } } peers { permanent_uuid: "3c0f82bda34740a1b9c018904f6b6eee" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44807 } }
I20260807 10:49:38.614434 4222 catalog_manager.cc:5495] T 10b673962a024d6abc7ef41c9cbae32e P 3c0f82bda34740a1b9c018904f6b6eee reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c0f82bda34740a1b9c018904f6b6eee (127.3.73.193). New cstate: current_term: 1 leader_uuid: "3c0f82bda34740a1b9c018904f6b6eee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fd8ee8288a41f49b575ab308cd968d" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 40471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1def62b5dd654258b4249815c6b149bc" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c0f82bda34740a1b9c018904f6b6eee" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44807 } health_report { overall_health: HEALTHY } } }
W20260807 10:49:38.635983 4713 tablet.cc:2369] T 10b673962a024d6abc7ef41c9cbae32e P 1def62b5dd654258b4249815c6b149bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 10:49:38 dist-test-slave-x5fp krb5kdc[4159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.193@KRBTEST.COM
I20260807 10:49:38.670063 4947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:38.648032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37588 (local address 127.3.73.193:44807)
0807 10:49:38.648386 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:38.648393 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:38.649703 (+ 1310us) server_negotiation.cc:407] Connection header received
0807 10:49:38.649902 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:38.649906 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:38.649977 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:38.650059 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:38.652595 (+ 2536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:38.653580 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:38.655326 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:38.655825 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:38.655930 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:38.661386 (+ 5456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:38.661404 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:38.661429 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:38.661485 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:38.666141 (+ 4656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:38.667100 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:38.667109 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:38.667112 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:38.667482 (+ 370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:38.668017 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:38.668021 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:38.668023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:38.668528 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:38.668896 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:38.669196 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 10:49:38.669487 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
Aug 07 10:49:38 dist-test-slave-x5fp krb5kdc[4159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099778, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 10:49:38 dist-test-slave-x5fp krb5kdc[4159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099778, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.73.193@KRBTEST.COM
I20260807 10:49:38.744899 4947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:38.725506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37596 (local address 127.3.73.193:44807)
0807 10:49:38.725841 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:38.725849 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:38.726246 (+ 397us) server_negotiation.cc:407] Connection header received
0807 10:49:38.726391 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:38.726396 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:38.726470 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:38.726624 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:38.728483 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:38.729532 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:38.731156 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:38.731780 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:38.731935 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:38.736783 (+ 4848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:38.736791 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:38.736796 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:38.736850 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:38.741266 (+ 4416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:38.742126 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:38.742139 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:38.742142 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:38.742338 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:38.742779 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:38.742787 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:38.742790 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:38.743241 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:38.743696 (+ 455us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:38.744027 (+ 331us) server_negotiation.cc:299] Negotiation successful
0807 10:49:38.744367 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
Aug 07 10:49:38 dist-test-slave-x5fp krb5kdc[4159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 10:49:38 dist-test-slave-x5fp krb5kdc[4159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.193@KRBTEST.COM
I20260807 10:49:38.811544 4947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:38.794242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37612 (local address 127.3.73.193:44807)
0807 10:49:38.794739 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:38.794752 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:38.795192 (+ 440us) server_negotiation.cc:407] Connection header received
0807 10:49:38.795446 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:38.795451 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:38.795541 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:38.795669 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:38.798071 (+ 2402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:38.798989 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:38.800955 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:38.801484 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:38.801665 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:38.805912 (+ 4247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:38.805918 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:38.805922 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:38.805964 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:38.808968 (+ 3004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:38.809767 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:38.809772 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:38.809783 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:38.809933 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:38.810315 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:38.810319 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:38.810321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:38.810609 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:38.810853 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:38.811090 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 10:49:38.811266 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256}
W20260807 10:49:38.814015 4388 tablet_service.cc:2908] Rejecting scan request for tablet 10b673962a024d6abc7ef41c9cbae32e: Uninitialized: clean time has not yet been initialized
I20260807 10:49:38.817927 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 4314
I20260807 10:49:38.836081 4947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:38.819042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37614 (local address 127.3.73.193:44807)
0807 10:49:38.819294 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:38.819306 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:38.819800 (+ 494us) server_negotiation.cc:407] Connection header received
0807 10:49:38.819918 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:38.819922 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:38.819992 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:38.820079 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:38.821864 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:38.822638 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:38.824831 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:38.825603 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:38.825755 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:38.828791 (+ 3036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:38.828799 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:38.828805 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:38.828857 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:38.832403 (+ 3546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:38.833494 (+ 1091us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:38.833499 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:38.833501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:38.833665 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:38.834310 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:38.834314 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:38.834317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:38.834670 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:38.835039 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:38.835390 (+ 351us) server_negotiation.cc:299] Negotiation successful
0807 10:49:38.835644 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 10:49:38.837181 4503 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:49:38.934937 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 4314
I20260807 10:49:38.967634 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 4521
Aug 07 10:49:38 dist-test-slave-x5fp krb5kdc[4159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.194@KRBTEST.COM
I20260807 10:49:38.984395 4946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:38.968553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42094 (local address 127.3.73.194:46031)
0807 10:49:38.968824 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:38.968834 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:38.968860 (+ 26us) server_negotiation.cc:407] Connection header received
0807 10:49:38.969066 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:38.969071 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:38.969151 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:38.969246 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:38.970628 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:38.971432 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:38.973036 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:38.973462 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:38.973571 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:38.978375 (+ 4804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:38.978391 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:38.978410 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:38.978451 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:38.981509 (+ 3058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:38.982401 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:38.982410 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:38.982412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:38.982561 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:38.982964 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:38.982968 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:38.982970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:38.983357 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:38.983594 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:38.983938 (+ 344us) server_negotiation.cc:299] Negotiation successful
0807 10:49:38.984106 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 10:49:38.985318 4705 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:49:39.077210 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 4521
I20260807 10:49:39.103703 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 4719
Aug 07 10:49:39 dist-test-slave-x5fp krb5kdc[4159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099778, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.195@KRBTEST.COM
I20260807 10:49:39.121999 4988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:39.104678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49312 (local address 127.3.73.195:40471)
0807 10:49:39.105254 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:39.105259 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:39.105277 (+ 18us) server_negotiation.cc:407] Connection header received
0807 10:49:39.105336 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:39.105340 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:39.105413 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:39.105506 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:39.107077 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:39.107854 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:39.109576 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:39.110091 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:39.110236 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:39.115097 (+ 4861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:39.115114 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:39.115132 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:39.115176 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:39.118753 (+ 3577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:39.119561 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:39.119570 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:39.119573 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:39.119771 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:39.120221 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:39.120225 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:39.120227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:39.120627 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:39.121030 (+ 403us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:39.121342 (+ 312us) server_negotiation.cc:299] Negotiation successful
0807 10:49:39.121537 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":326,"threads_started":1}
I20260807 10:49:39.122784 4917 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:49:39.215752 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 4719
I20260807 10:49:39.243670 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 4177
I20260807 10:49:39.244966 4275 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:49:39.358006 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 4177
2026-08-07T10:49:39Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (5068 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:49:39 dist-test-slave-x5fp krb5kdc[4996](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:49:39 dist-test-slave-x5fp krb5kdc[4996](info): set up 2 sockets
Aug 07 10:49:39 dist-test-slave-x5fp krb5kdc[4996](info): commencing operation
krb5kdc: starting...
W20260807 10:49:40.727972 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.314s user 0.001s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:49:40 dist-test-slave-x5fp krb5kdc[4996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:49:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:49:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.73.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
I20260807 10:49:40.990597 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:41707
--webserver_interface=127.3.73.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:34965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:41707
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:49:41.352212 5014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:49:41.352665 5014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:49:41.352849 5014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:49:41.360455 5014 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:49:41.360641 5014 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 10:49:41.360769 5014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:49:41.360909 5014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:49:41.361002 5014 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:49:41.361099 5014 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:49:41.371627 5014 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:34965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:41707
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:41707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:49:41.373153 5014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:49:41.374646 5014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:49:41.387914 5020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:41.387925 5023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:41.387925 5026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:49:41.389604 5014 server_base.cc:1034] running on GCE node
I20260807 10:49:41.390625 5014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:49:41.392469 5014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:49:41.393811 5014 hybrid_clock.cc:648] HybridClock initialized: now 1786099781393702 us; error 52 us; skew 500 ppm
I20260807 10:49:41.394228 5014 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:49:41 dist-test-slave-x5fp krb5kdc[4996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099781, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:49:41.403484 5014 init.cc:377] Logged in from keytab as kudu/127.3.73.254@KRBTEST.COM (short username kudu)
I20260807 10:49:41.405583 5014 webserver.cc:460] Webserver started at http://127.3.73.254:34099/ using document root <none> and password file <none>
I20260807 10:49:41.406438 5014 fs_manager.cc:359] Metadata directory not provided
I20260807 10:49:41.406538 5014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:49:41.406847 5014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:49:41.410228 5014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "9a4a5d57a3a44ef6b4d184ffd17c4fd7"
format_stamp: "Formatted at 2026-08-07 10:49:41 on dist-test-slave-x5fp"
server_key: "34f108d8c7bb05fee130833d709e5488"
server_key_iv: "884336c91ed913fd094963a1f0a290e2"
server_key_version: "encryptionkey@0"
I20260807 10:49:41.411091 5014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "9a4a5d57a3a44ef6b4d184ffd17c4fd7"
format_stamp: "Formatted at 2026-08-07 10:49:41 on dist-test-slave-x5fp"
server_key: "34f108d8c7bb05fee130833d709e5488"
server_key_iv: "884336c91ed913fd094963a1f0a290e2"
server_key_version: "encryptionkey@0"
I20260807 10:49:41.416673 5014 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 10:49:41.420481 5038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:41.422161 5014 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 10:49:41.422395 5014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "9a4a5d57a3a44ef6b4d184ffd17c4fd7"
format_stamp: "Formatted at 2026-08-07 10:49:41 on dist-test-slave-x5fp"
server_key: "34f108d8c7bb05fee130833d709e5488"
server_key_iv: "884336c91ed913fd094963a1f0a290e2"
server_key_version: "encryptionkey@0"
I20260807 10:49:41.422677 5014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:49:41.452996 5014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:49:41.458207 5014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:49:41.458559 5014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:49:41.485447 5014 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:41707
I20260807 10:49:41.485472 5125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:41707 every 8 connection(s)
I20260807 10:49:41.487040 5014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:49:41.487677 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 5014
I20260807 10:49:41.487970 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:49:41.488935 3367 external_mini_cluster.cc:1428] Setting key 1edb22f2ed912fd4cb1aa9175ab47ea2
I20260807 10:49:41.492491 5127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 10:49:41 dist-test-slave-x5fp krb5kdc[4996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:49:41.508070 5127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a4a5d57a3a44ef6b4d184ffd17c4fd7: Bootstrap starting.
I20260807 10:49:41.512863 5127 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a4a5d57a3a44ef6b4d184ffd17c4fd7: Neither blocks nor log segments found. Creating new log.
I20260807 10:49:41.514287 5127 log.cc:824] T 00000000000000000000000000000000 P 9a4a5d57a3a44ef6b4d184ffd17c4fd7: Log is configured to *not* fsync() on all Append() calls
I20260807 10:49:41.518383 5127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a4a5d57a3a44ef6b4d184ffd17c4fd7: No bootstrap required, opened a new log
I20260807 10:49:41.518548 5130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:41.491202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53114 (local address 127.3.73.254:41707)
0807 10:49:41.492262 (+ 1060us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:41.492288 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:41.492348 (+ 60us) server_negotiation.cc:407] Connection header received
0807 10:49:41.495111 (+ 2763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:41.495147 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:41.495741 (+ 594us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:41.496339 (+ 598us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:41.498052 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:41.500977 (+ 2925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:41.502611 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:41.503700 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:41.503909 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:41.508473 (+ 4564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:41.508486 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:41.508508 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:41.508568 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:41.512955 (+ 4387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:41.514012 (+ 1057us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:41.514020 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:41.514028 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:41.514281 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:41.514825 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:41.514830 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:41.514832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:41.515537 (+ 705us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:41.515847 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:41.516396 (+ 549us) server_negotiation.cc:299] Negotiation successful
0807 10:49:41.516979 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":800,"thread_start_us":421,"threads_started":1}
I20260807 10:49:41.527733 5127 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a4a5d57a3a44ef6b4d184ffd17c4fd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a4a5d57a3a44ef6b4d184ffd17c4fd7" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41707 } }
I20260807 10:49:41.528136 5127 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a4a5d57a3a44ef6b4d184ffd17c4fd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:49:41.528216 5127 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a4a5d57a3a44ef6b4d184ffd17c4fd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a4a5d57a3a44ef6b4d184ffd17c4fd7, State: Initialized, Role: FOLLOWER
I20260807 10:49:41.528988 5127 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a4a5d57a3a44ef6b4d184ffd17c4fd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a4a5d57a3a44ef6b4d184ffd17c4fd7" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41707 } }
I20260807 10:49:41.529237 5127 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a4a5d57a3a44ef6b4d184ffd17c4fd7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:49:41.529464 5127 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a4a5d57a3a44ef6b4d184ffd17c4fd7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:49:41.529673 5127 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a4a5d57a3a44ef6b4d184ffd17c4fd7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:49:41.532160 5127 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a4a5d57a3a44ef6b4d184ffd17c4fd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a4a5d57a3a44ef6b4d184ffd17c4fd7" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41707 } }
I20260807 10:49:41.533166 5127 leader_election.cc:302] T 00000000000000000000000000000000 P 9a4a5d57a3a44ef6b4d184ffd17c4fd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a4a5d57a3a44ef6b4d184ffd17c4fd7; no voters:
I20260807 10:49:41.533912 5127 leader_election.cc:288] T 00000000000000000000000000000000 P 9a4a5d57a3a44ef6b4d184ffd17c4fd7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:49:41.534137 5132 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a4a5d57a3a44ef6b4d184ffd17c4fd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:49:41.534971 5132 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a4a5d57a3a44ef6b4d184ffd17c4fd7 [term 1 LEADER]: Becoming Leader. State: Replica: 9a4a5d57a3a44ef6b4d184ffd17c4fd7, State: Running, Role: LEADER
I20260807 10:49:41.535655 5132 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a4a5d57a3a44ef6b4d184ffd17c4fd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a4a5d57a3a44ef6b4d184ffd17c4fd7" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41707 } }
I20260807 10:49:41.536420 5127 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a4a5d57a3a44ef6b4d184ffd17c4fd7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:49:41.539426 5133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a4a5d57a3a44ef6b4d184ffd17c4fd7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a4a5d57a3a44ef6b4d184ffd17c4fd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a4a5d57a3a44ef6b4d184ffd17c4fd7" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41707 } } }
I20260807 10:49:41.539832 5135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a4a5d57a3a44ef6b4d184ffd17c4fd7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a4a5d57a3a44ef6b4d184ffd17c4fd7. Latest consensus state: current_term: 1 leader_uuid: "9a4a5d57a3a44ef6b4d184ffd17c4fd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a4a5d57a3a44ef6b4d184ffd17c4fd7" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41707 } } }
I20260807 10:49:41.540136 5135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a4a5d57a3a44ef6b4d184ffd17c4fd7 [sys.catalog]: This master's current role is: LEADER
I20260807 10:49:41.540159 5133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a4a5d57a3a44ef6b4d184ffd17c4fd7 [sys.catalog]: This master's current role is: LEADER
I20260807 10:49:41.543185 5148 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:49:41.548192 5148 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:49:41.557971 5148 catalog_manager.cc:1347] Generated new cluster ID: 833c95424b1042dc9df9d8482607e166
I20260807 10:49:41.558125 5148 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:49:41.566200 5148 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:49:41.568080 5148 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:49:41.576591 5148 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a4a5d57a3a44ef6b4d184ffd17c4fd7: Generated new TSK 0
I20260807 10:49:41.577562 5148 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.73.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab.
Entry for principal kudu/127.3.73.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab.
I20260807 10:49:41.646167 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.193:0
--local_ip_for_outbound_sockets=127.3.73.193
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:41707
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:34965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:49:42.013111 5164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:49:42.013447 5164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:49:42.013617 5164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:49:42.021471 5164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:49:42.021591 5164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:49:42.021775 5164 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 10:49:42.021872 5164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.193
I20260807 10:49:42.032464 5164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:34965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.73.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.73.254:41707
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.193
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:49:42.033994 5164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:49:42.035583 5164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:49:42.052330 5171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:42.052330 5172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:42.052906 5175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:49:42.053155 5164 server_base.cc:1034] running on GCE node
I20260807 10:49:42.054203 5164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:49:42.055375 5164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:49:42.056869 5164 hybrid_clock.cc:648] HybridClock initialized: now 1786099782056685 us; error 52 us; skew 500 ppm
I20260807 10:49:42.057225 5164 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:49:42 dist-test-slave-x5fp krb5kdc[4996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099782, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:49:42.060779 5164 init.cc:377] Logged in from keytab as kudu/127.3.73.193@KRBTEST.COM (short username kudu)
I20260807 10:49:42.062505 5164 webserver.cc:460] Webserver started at http://127.3.73.193:41967/ using document root <none> and password file <none>
I20260807 10:49:42.063241 5164 fs_manager.cc:359] Metadata directory not provided
I20260807 10:49:42.063340 5164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:49:42.063633 5164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:49:42.066928 5164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-0/data/instance:
uuid: "ec508c2f08974d0b8c83a369c0d40616"
format_stamp: "Formatted at 2026-08-07 10:49:42 on dist-test-slave-x5fp"
server_key: "6f72c5df33b61caebe7d63b69aa84254"
server_key_iv: "66af90f2f436ef6f60ec39a59f0d5bfe"
server_key_version: "encryptionkey@0"
I20260807 10:49:42.067744 5164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance:
uuid: "ec508c2f08974d0b8c83a369c0d40616"
format_stamp: "Formatted at 2026-08-07 10:49:42 on dist-test-slave-x5fp"
server_key: "6f72c5df33b61caebe7d63b69aa84254"
server_key_iv: "66af90f2f436ef6f60ec39a59f0d5bfe"
server_key_version: "encryptionkey@0"
I20260807 10:49:42.073968 5164 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 10:49:42.078629 5184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:42.080211 5164 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 10:49:42.080412 5164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-0/wal
uuid: "ec508c2f08974d0b8c83a369c0d40616"
format_stamp: "Formatted at 2026-08-07 10:49:42 on dist-test-slave-x5fp"
server_key: "6f72c5df33b61caebe7d63b69aa84254"
server_key_iv: "66af90f2f436ef6f60ec39a59f0d5bfe"
server_key_version: "encryptionkey@0"
I20260807 10:49:42.080648 5164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:49:42.114816 5164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:49:42.120056 5164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:49:42.120388 5164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:49:42.123603 5164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:49:42.123723 5164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:42.123885 5164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:49:42.123965 5164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 10:49:42 dist-test-slave-x5fp krb5kdc[4996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099782, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:49:42.151564 5195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:42.124471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:42835 (local address 127.3.73.254:41707)
0807 10:49:42.125144 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:42.125150 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:42.126227 (+ 1077us) server_negotiation.cc:407] Connection header received
0807 10:49:42.128691 (+ 2464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:42.128697 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:42.128815 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:42.128957 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:42.131326 (+ 2369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:42.132451 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:42.135711 (+ 3260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:42.136687 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:42.136899 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:42.142343 (+ 5444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:42.142355 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:42.142363 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:42.142425 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:42.147018 (+ 4593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:42.148219 (+ 1201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:42.148226 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:42.148230 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:42.148387 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:42.149100 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:42.149106 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:42.149113 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:42.149645 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:42.149847 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:42.150639 (+ 792us) server_negotiation.cc:299] Negotiation successful
0807 10:49:42.150971 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":416,"threads_started":1}
I20260807 10:49:42.152230 5196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:42.125084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41707 (local address 127.3.73.193:42835)
0807 10:49:42.126030 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:42.126073 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:42.128321 (+ 2248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:42.129223 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:42.129233 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:42.129926 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:49:42.131096 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:42.131131 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:42.132642 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:42.132647 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:42.135467 (+ 2820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:42.135492 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:42.137005 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:42.137011 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:42.137268 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:42.138288 (+ 1020us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:49:42.138334 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:49:42.142009 (+ 3675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:49:42.147242 (+ 5233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:42.147255 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:42.147279 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:42.148059 (+ 780us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:42.148533 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:42.148537 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:42.148540 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:42.148896 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:42.149811 (+ 915us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:49:42.149818 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:49:42.150197 (+ 379us) client_negotiation.cc:769] Sending connection context
0807 10:49:42.150438 (+ 241us) client_negotiation.cc:240] Negotiation successful
0807 10:49:42.150886 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"thread_start_us":391,"threads_started":1}
I20260807 10:49:42.181800 5164 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.193:44329
I20260807 10:49:42.181903 5370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.193:44329 every 8 connection(s)
I20260807 10:49:42.183470 5164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
I20260807 10:49:42.183981 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 5164
I20260807 10:49:42.184216 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance
I20260807 10:49:42.184451 5195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:42.164544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:35951 (local address 127.3.73.254:41707)
0807 10:49:42.164823 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:42.164832 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:42.166044 (+ 1212us) server_negotiation.cc:407] Connection header received
0807 10:49:42.166324 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:42.166333 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:42.166438 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:42.166563 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:42.168889 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:42.169980 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:42.172615 (+ 2635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:42.173195 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:42.173359 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:42.176912 (+ 3553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:42.176921 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:42.176928 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:42.176985 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:42.180883 (+ 3898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:42.182072 (+ 1189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:42.182079 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:42.182082 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:42.182256 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:42.182680 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:42.182685 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:42.182689 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:42.183020 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:42.183189 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:42.183695 (+ 506us) server_negotiation.cc:299] Negotiation successful
0807 10:49:42.183960 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 10:49:42.184643 5306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:42.164468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41707 (local address 127.3.73.193:35951)
0807 10:49:42.165874 (+ 1406us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:42.165899 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:42.166137 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:42.166631 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:42.166639 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:42.167683 (+ 1044us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:49:42.168577 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:42.168600 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:42.170164 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:42.170169 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:42.172402 (+ 2233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:42.172442 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:42.173506 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:42.173513 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:42.173778 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:42.174661 (+ 883us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:49:42.174697 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:49:42.176511 (+ 1814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:49:42.181077 (+ 4566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:42.181082 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:42.181084 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:42.181857 (+ 773us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:42.182417 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:42.182421 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:42.182423 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:42.182553 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:42.183150 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:49:42.183155 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:49:42.183359 (+ 204us) client_negotiation.cc:769] Sending connection context
0807 10:49:42.183525 (+ 166us) client_negotiation.cc:240] Negotiation successful
0807 10:49:42.183993 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1165,"thread_start_us":984,"threads_started":1}
I20260807 10:49:42.185133 3367 external_mini_cluster.cc:1428] Setting key 4558eff5199c36849457499cb082687e
I20260807 10:49:42.188831 5371 heartbeater.cc:347] Connected to a master server at 127.3.73.254:41707
I20260807 10:49:42.189133 5371 heartbeater.cc:464] Registering TS with master...
I20260807 10:49:42.190008 5371 heartbeater.cc:511] Master 127.3.73.254:41707 requested a full tablet report, sending...
I20260807 10:49:42.192528 5067 ts_manager.cc:194] Registered new tserver with Master: ec508c2f08974d0b8c83a369c0d40616 (127.3.73.193:44329)
I20260807 10:49:42.194737 5067 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.193@KRBTEST.COM'} at 127.3.73.193:42835
WARNING: no policy specified for kudu/127.3.73.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab.
Entry for principal kudu/127.3.73.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab.
I20260807 10:49:42.240829 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.194:0
--local_ip_for_outbound_sockets=127.3.73.194
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:41707
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:34965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:49:42.604229 5376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:49:42.604548 5376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:49:42.604763 5376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:49:42.612412 5376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:49:42.612550 5376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:49:42.612699 5376 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 10:49:42.612839 5376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.194
I20260807 10:49:42.622891 5376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:34965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.73.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.73.254:41707
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.194
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:49:42.624313 5376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:49:42.625813 5376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:49:42.640117 5389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:42.640164 5385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:42.640193 5386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:49:42.640956 5376 server_base.cc:1034] running on GCE node
I20260807 10:49:42.641788 5376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:49:42.642930 5376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:49:42.644219 5376 hybrid_clock.cc:648] HybridClock initialized: now 1786099782644124 us; error 64 us; skew 500 ppm
I20260807 10:49:42.644558 5376 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:49:42 dist-test-slave-x5fp krb5kdc[4996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099782, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:49:42.647964 5376 init.cc:377] Logged in from keytab as kudu/127.3.73.194@KRBTEST.COM (short username kudu)
I20260807 10:49:42.649528 5376 webserver.cc:460] Webserver started at http://127.3.73.194:44921/ using document root <none> and password file <none>
I20260807 10:49:42.650243 5376 fs_manager.cc:359] Metadata directory not provided
I20260807 10:49:42.650336 5376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:49:42.650617 5376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:49:42.653772 5376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-1/data/instance:
uuid: "59c8fcaf1fd44e3d8fb950f658162cb0"
format_stamp: "Formatted at 2026-08-07 10:49:42 on dist-test-slave-x5fp"
server_key: "e06b7a52c56eb05671aa88394270119b"
server_key_iv: "ab6f33722fb08b09272982ef60abb2da"
server_key_version: "encryptionkey@0"
I20260807 10:49:42.654651 5376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance:
uuid: "59c8fcaf1fd44e3d8fb950f658162cb0"
format_stamp: "Formatted at 2026-08-07 10:49:42 on dist-test-slave-x5fp"
server_key: "e06b7a52c56eb05671aa88394270119b"
server_key_iv: "ab6f33722fb08b09272982ef60abb2da"
server_key_version: "encryptionkey@0"
I20260807 10:49:42.660146 5376 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 10:49:42.664063 5400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:42.665583 5376 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 10:49:42.665719 5376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-1/wal
uuid: "59c8fcaf1fd44e3d8fb950f658162cb0"
format_stamp: "Formatted at 2026-08-07 10:49:42 on dist-test-slave-x5fp"
server_key: "e06b7a52c56eb05671aa88394270119b"
server_key_iv: "ab6f33722fb08b09272982ef60abb2da"
server_key_version: "encryptionkey@0"
I20260807 10:49:42.665907 5376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:49:42.700098 5376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:49:42.704877 5376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:49:42.705127 5376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:49:42.707892 5376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:49:42.707985 5376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:42.708195 5376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:49:42.708273 5376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 10:49:42 dist-test-slave-x5fp krb5kdc[4996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099782, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:49:42.736789 5412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:42.709001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:41261 (local address 127.3.73.254:41707)
0807 10:49:42.709510 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:42.709517 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:42.710892 (+ 1375us) server_negotiation.cc:407] Connection header received
0807 10:49:42.714388 (+ 3496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:42.714394 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:42.714503 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:42.714639 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:42.717704 (+ 3065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:42.718741 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:42.722227 (+ 3486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:42.723041 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:42.723179 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:42.729258 (+ 6079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:42.729264 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:42.729268 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:42.729308 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:42.732112 (+ 2804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:42.733487 (+ 1375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:42.733492 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:42.733494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:42.733701 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:42.734422 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:42.734430 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:42.734435 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:42.734800 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:42.734982 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:42.736020 (+ 1038us) server_negotiation.cc:299] Negotiation successful
0807 10:49:42.736368 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":273,"threads_started":1}
I20260807 10:49:42.738669 5413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:42.709731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41707 (local address 127.3.73.194:41261)
0807 10:49:42.710669 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:42.710733 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:42.713886 (+ 3153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:42.714935 (+ 1049us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:42.714951 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:42.716139 (+ 1188us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:49:42.717420 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:42.717462 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:42.718964 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:42.718972 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:42.721988 (+ 3016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:42.722008 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:42.723653 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:42.723669 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:42.724101 (+ 432us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:42.725213 (+ 1112us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:49:42.725246 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:49:42.728938 (+ 3692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:49:42.732314 (+ 3376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:42.732328 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:42.732354 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:42.733289 (+ 935us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:42.733852 (+ 563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:42.733859 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:42.733870 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:42.734236 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:42.734954 (+ 718us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:49:42.734963 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:49:42.735515 (+ 552us) client_negotiation.cc:769] Sending connection context
0807 10:49:42.735916 (+ 401us) client_negotiation.cc:240] Negotiation successful
0807 10:49:42.736701 (+ 785us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":392,"threads_started":1}
I20260807 10:49:42.763001 5518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:42.746500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41707 (local address 127.3.73.194:45585)
0807 10:49:42.747231 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:42.747266 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:42.747454 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:42.747815 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:42.747820 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:42.748565 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:49:42.749120 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:42.749138 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:42.750842 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:42.750852 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:42.752638 (+ 1786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:42.752651 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:42.753716 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:42.753722 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:42.753896 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:42.754627 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:49:42.754652 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:49:42.756215 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:49:42.759711 (+ 3496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:42.759720 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:42.759726 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:42.760500 (+ 774us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:42.760973 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:42.760982 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:42.760986 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:42.761182 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:42.762081 (+ 899us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:49:42.762085 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:49:42.762247 (+ 162us) client_negotiation.cc:769] Sending connection context
0807 10:49:42.762381 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 10:49:42.762631 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":424,"threads_started":1}
I20260807 10:49:42.763334 5412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:42.746435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:45585 (local address 127.3.73.254:41707)
0807 10:49:42.746645 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:42.746651 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:42.747372 (+ 721us) server_negotiation.cc:407] Connection header received
0807 10:49:42.747592 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:42.747597 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:42.747677 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:42.747783 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:42.749345 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:42.750666 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:42.752938 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:42.753466 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:42.753622 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:42.756466 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:42.756475 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:42.756483 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:42.756568 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:42.759566 (+ 2998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:42.760643 (+ 1077us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:42.760647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:42.760650 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:42.760847 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:42.761358 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:42.761366 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:42.761372 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:42.761948 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:42.762142 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:42.762547 (+ 405us) server_negotiation.cc:299] Negotiation successful
0807 10:49:42.762858 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 10:49:42.770362 5376 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.194:34367
I20260807 10:49:42.770445 5578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.194:34367 every 8 connection(s)
I20260807 10:49:42.771685 5376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
I20260807 10:49:42.773984 5580 heartbeater.cc:347] Connected to a master server at 127.3.73.254:41707
I20260807 10:49:42.774271 5580 heartbeater.cc:464] Registering TS with master...
I20260807 10:49:42.774969 5580 heartbeater.cc:511] Master 127.3.73.254:41707 requested a full tablet report, sending...
I20260807 10:49:42.776522 5067 ts_manager.cc:194] Registered new tserver with Master: 59c8fcaf1fd44e3d8fb950f658162cb0 (127.3.73.194:34367)
I20260807 10:49:42.777421 5067 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.194@KRBTEST.COM'} at 127.3.73.194:41261
I20260807 10:49:42.778793 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 5376
I20260807 10:49:42.779013 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance
I20260807 10:49:42.779605 3367 external_mini_cluster.cc:1428] Setting key ca415078ef449a7c5b80a213685a3bb1
WARNING: no policy specified for kudu/127.3.73.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab.
Entry for principal kudu/127.3.73.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab.
I20260807 10:49:42.832983 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.195:0
--local_ip_for_outbound_sockets=127.3.73.195
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:41707
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:34965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 10:49:43.198462 5371 heartbeater.cc:503] Master 127.3.73.254:41707 was elected leader, sending a full tablet report...
W20260807 10:49:43.201135 5585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:49:43.201618 5585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:49:43.201862 5585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:49:43.211390 5585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:49:43.211596 5585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:49:43.211771 5585 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 10:49:43.211928 5585 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.195
I20260807 10:49:43.222711 5585 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:34965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.73.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.73.254:41707
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.195
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:49:43.224260 5585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:49:43.226207 5585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:49:43.243378 5593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:43.243945 5592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:43.243990 5597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:49:43.246546 5585 server_base.cc:1034] running on GCE node
I20260807 10:49:43.247447 5585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:49:43.248497 5585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:49:43.249761 5585 hybrid_clock.cc:648] HybridClock initialized: now 1786099783249659 us; error 92 us; skew 500 ppm
I20260807 10:49:43.250104 5585 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:49:43 dist-test-slave-x5fp krb5kdc[4996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099783, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:49:43.253613 5585 init.cc:377] Logged in from keytab as kudu/127.3.73.195@KRBTEST.COM (short username kudu)
I20260807 10:49:43.255367 5585 webserver.cc:460] Webserver started at http://127.3.73.195:38083/ using document root <none> and password file <none>
I20260807 10:49:43.256636 5585 fs_manager.cc:359] Metadata directory not provided
I20260807 10:49:43.256794 5585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:49:43.257135 5585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:49:43.260490 5585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-2/data/instance:
uuid: "1c06edc993584b45a6fd20b3d6cb4ded"
format_stamp: "Formatted at 2026-08-07 10:49:43 on dist-test-slave-x5fp"
server_key: "0a38ec73205f507f9f4d2c2277ffa8b6"
server_key_iv: "336c14d3913f55f89f38eb365a961033"
server_key_version: "encryptionkey@0"
I20260807 10:49:43.261508 5585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance:
uuid: "1c06edc993584b45a6fd20b3d6cb4ded"
format_stamp: "Formatted at 2026-08-07 10:49:43 on dist-test-slave-x5fp"
server_key: "0a38ec73205f507f9f4d2c2277ffa8b6"
server_key_iv: "336c14d3913f55f89f38eb365a961033"
server_key_version: "encryptionkey@0"
I20260807 10:49:43.267464 5585 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 10:49:43.271601 5606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:43.273201 5585 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 10:49:43.273411 5585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-2/wal
uuid: "1c06edc993584b45a6fd20b3d6cb4ded"
format_stamp: "Formatted at 2026-08-07 10:49:43 on dist-test-slave-x5fp"
server_key: "0a38ec73205f507f9f4d2c2277ffa8b6"
server_key_iv: "336c14d3913f55f89f38eb365a961033"
server_key_version: "encryptionkey@0"
I20260807 10:49:43.273674 5585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:49:43.297056 5585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:49:43.301999 5585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:49:43.302322 5585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:49:43.305830 5585 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:49:43.305956 5585 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:43.306105 5585 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:49:43.306195 5585 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 10:49:43 dist-test-slave-x5fp krb5kdc[4996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099783, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:49:43.333884 5620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:43.306495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:33283 (local address 127.3.73.254:41707)
0807 10:49:43.307308 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:43.307313 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:43.308754 (+ 1441us) server_negotiation.cc:407] Connection header received
0807 10:49:43.311081 (+ 2327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:43.311087 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:43.311194 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:43.311511 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:43.314436 (+ 2925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:43.315494 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:43.318358 (+ 2864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:43.319030 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:43.319199 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:43.325346 (+ 6147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:43.325353 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:43.325356 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:43.325400 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:43.329365 (+ 3965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:43.330565 (+ 1200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:43.330573 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:43.330577 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:43.330918 (+ 341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:43.331564 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:43.331571 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:43.331575 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:43.331928 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:43.332140 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:43.333148 (+ 1008us) server_negotiation.cc:299] Negotiation successful
0807 10:49:43.333429 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":700,"thread_start_us":582,"threads_started":1}
I20260807 10:49:43.335530 5621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:43.307271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41707 (local address 127.3.73.195:33283)
0807 10:49:43.308333 (+ 1062us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:43.308391 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:43.310731 (+ 2340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:43.311859 (+ 1128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:43.311874 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:43.313090 (+ 1216us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:49:43.314201 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:43.314226 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:43.315685 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:43.315692 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:43.318057 (+ 2365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:43.318081 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:43.319564 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:43.319570 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:43.319860 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:43.320825 (+ 965us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:49:43.320893 (+ 68us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:49:43.324994 (+ 4101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:49:43.329567 (+ 4573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:43.329579 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:43.329613 (+ 34us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:43.330391 (+ 778us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:43.331101 (+ 710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:43.331105 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:43.331108 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:43.331408 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:43.332115 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:49:43.332126 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:49:43.332598 (+ 472us) client_negotiation.cc:769] Sending connection context
0807 10:49:43.333040 (+ 442us) client_negotiation.cc:240] Negotiation successful
0807 10:49:43.333722 (+ 682us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"thread_start_us":410,"threads_started":1}
I20260807 10:49:43.364437 5620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:43.344689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:34467 (local address 127.3.73.254:41707)
0807 10:49:43.344978 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:43.344988 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:43.346083 (+ 1095us) server_negotiation.cc:407] Connection header received
0807 10:49:43.346374 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:43.346379 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:43.346459 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:43.346613 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:43.348973 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:43.349857 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:43.352549 (+ 2692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:43.353114 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:43.353308 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:43.356864 (+ 3556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:43.356870 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:43.356873 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:43.356912 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:43.360533 (+ 3621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:43.361638 (+ 1105us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:43.361647 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:43.361653 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:43.361912 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:43.362458 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:43.362462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:43.362464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:43.362765 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:43.362968 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:43.363649 (+ 681us) server_negotiation.cc:299] Negotiation successful
0807 10:49:43.363977 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260807 10:49:43.364948 5712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:43.344626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41707 (local address 127.3.73.195:34467)
0807 10:49:43.345687 (+ 1061us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:43.345717 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:43.346175 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:43.346664 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:43.346671 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:43.347783 (+ 1112us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:49:43.348675 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:43.348702 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:43.350088 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:43.350098 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:43.352321 (+ 2223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:43.352344 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:43.353342 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:43.353356 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:43.353648 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:43.354790 (+ 1142us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:49:43.354835 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:49:43.356520 (+ 1685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:49:43.360708 (+ 4188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:43.360713 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:43.360752 (+ 39us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:43.361445 (+ 693us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:43.362081 (+ 636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:43.362089 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:43.362096 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:43.362319 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:43.362932 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:49:43.362941 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:49:43.363262 (+ 321us) client_negotiation.cc:769] Sending connection context
0807 10:49:43.363493 (+ 231us) client_negotiation.cc:240] Negotiation successful
0807 10:49:43.364077 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":814,"thread_start_us":594,"threads_started":1}
I20260807 10:49:43.378813 5585 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.195:34257
I20260807 10:49:43.378890 5782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.195:34257 every 8 connection(s)
I20260807 10:49:43.380766 5585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
I20260807 10:49:43.383580 5783 heartbeater.cc:347] Connected to a master server at 127.3.73.254:41707
I20260807 10:49:43.383982 5783 heartbeater.cc:464] Registering TS with master...
I20260807 10:49:43.384896 5783 heartbeater.cc:511] Master 127.3.73.254:41707 requested a full tablet report, sending...
I20260807 10:49:43.386759 5067 ts_manager.cc:194] Registered new tserver with Master: 1c06edc993584b45a6fd20b3d6cb4ded (127.3.73.195:34257)
I20260807 10:49:43.387684 5067 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.195@KRBTEST.COM'} at 127.3.73.195:33283
I20260807 10:49:43.389472 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 5585
I20260807 10:49:43.389796 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance
I20260807 10:49:43.390431 3367 external_mini_cluster.cc:1428] Setting key 2012c6590a757a55b56706085dd5829c
I20260807 10:49:43.394130 3367 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 10:49:43 dist-test-slave-x5fp krb5kdc[4996](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099783, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 10:49:43 dist-test-slave-x5fp krb5kdc[4996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099783, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:49:43.465755 5620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:43.447197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53116 (local address 127.3.73.254:41707)
0807 10:49:43.447501 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:43.447512 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:43.448209 (+ 697us) server_negotiation.cc:407] Connection header received
0807 10:49:43.448408 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:43.448412 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:43.448480 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:43.448572 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:43.450920 (+ 2348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:43.451639 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:43.453664 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:43.454175 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:43.454277 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:43.459549 (+ 5272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:43.459553 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:43.459557 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:43.459597 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:43.462464 (+ 2867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:43.463378 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:43.463383 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:43.463385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:43.463548 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:43.464097 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:43.464100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:43.464103 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:43.464428 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:43.464629 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:43.465172 (+ 543us) server_negotiation.cc:299] Negotiation successful
0807 10:49:43.465361 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260807 10:49:43.470999 5067 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53116:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 10:49:43.478391 5067 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 10:49:43.523702 5803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:43.494553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49090 (local address 127.3.73.194:34367)
0807 10:49:43.507589 (+ 13036us) spinlock_profiling.cc:243] Waited 1.47 ms on lock 0x60c0000668c0. stack: 00007fcfb761566e 00007fcfb7615430 00007fcfb669c5ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007fcfca4fc775 00007fcfb765b1d3 00007fcfb765e91b 00007fcfb765e798 00000000009edb7c 00007fcfb7639eae 00007fcfba9136da 00007fcfb38a961e
0807 10:49:43.507790 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:43.507801 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:43.509649 (+ 1848us) server_negotiation.cc:407] Connection header received
0807 10:49:43.510136 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:43.510147 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:43.511023 (+ 876us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:43.511233 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:49:43.512957 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:43.514992 (+ 2035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:43.519063 (+ 4071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:43.520875 (+ 1812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:43.521067 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:43.521300 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:43.522992 (+ 1692us) server_negotiation.cc:299] Negotiation successful
0807 10:49:43.523219 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10529,"spinlock_wait_cycles":3389824,"thread_start_us":10389,"threads_started":1}
I20260807 10:49:43.523819 5805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:43.494632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50326 (local address 127.3.73.193:44329)
0807 10:49:43.508751 (+ 14119us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:43.508765 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:43.511099 (+ 2334us) server_negotiation.cc:407] Connection header received
0807 10:49:43.511224 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:43.511233 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:43.511464 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:43.511729 (+ 265us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:49:43.513202 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:43.514924 (+ 1722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:43.519238 (+ 4314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:43.521326 (+ 2088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:43.521466 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:43.521705 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:43.523042 (+ 1337us) server_negotiation.cc:299] Negotiation successful
0807 10:49:43.523284 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13877,"thread_start_us":13650,"threads_started":1}
I20260807 10:49:43.524452 5800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:43.493739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:34367 (local address 127.0.0.1:49090)
0807 10:49:43.509468 (+ 15729us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:43.509515 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:43.509839 (+ 324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:43.511578 (+ 1739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:43.511590 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:43.511628 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:49:43.512578 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:43.512602 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:43.515240 (+ 2638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:43.515246 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:43.518767 (+ 3521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:43.518791 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:43.521498 (+ 2707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:43.521506 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:43.521767 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:43.521834 (+ 67us) client_negotiation.cc:769] Sending connection context
0807 10:49:43.521995 (+ 161us) client_negotiation.cc:240] Negotiation successful
0807 10:49:43.522237 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15451,"mutex_wait_us":194,"thread_start_us":15242,"threads_started":1}
I20260807 10:49:43.525218 5802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:43.494089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:44329 (local address 127.0.0.1:50326)
0807 10:49:43.510748 (+ 16659us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:43.510769 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:43.510905 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:43.512004 (+ 1099us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:43.512010 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:43.512056 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:49:43.512893 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:43.512908 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:43.515154 (+ 2246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:43.515160 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:43.519020 (+ 3860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:43.519036 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:43.521938 (+ 2902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:43.521947 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:43.522146 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:43.522182 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 10:49:43.522264 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 10:49:43.522418 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16494,"mutex_wait_us":104,"thread_start_us":16326,"threads_started":1}
I20260807 10:49:43.530061 5476 tablet_service.cc:1467] Processing CreateTablet for tablet b56d404dbc234b46a49c4ebbdd972fe4 (DEFAULT_TABLE table=test-table [id=f03fc75a29e84902a4cad666426b547d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:49:43.530726 5263 tablet_service.cc:1467] Processing CreateTablet for tablet b56d404dbc234b46a49c4ebbdd972fe4 (DEFAULT_TABLE table=test-table [id=f03fc75a29e84902a4cad666426b547d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:49:43.531699 5476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b56d404dbc234b46a49c4ebbdd972fe4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:49:43.532640 5263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b56d404dbc234b46a49c4ebbdd972fe4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:49:43.538190 5800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:43.511989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:34257 (local address 127.0.0.1:46548)
0807 10:49:43.524800 (+ 12811us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:43.524824 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:43.524970 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:43.526434 (+ 1464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:43.526440 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:43.526475 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:49:43.526945 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:43.526962 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:43.530917 (+ 3955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:43.530924 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:43.533527 (+ 2603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:43.533547 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:43.536592 (+ 3045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:43.536600 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:43.537071 (+ 471us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:43.537116 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 10:49:43.537288 (+ 172us) client_negotiation.cc:240] Negotiation successful
0807 10:49:43.537574 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12620,"thread_start_us":14646,"threads_started":1}
I20260807 10:49:43.538551 5807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:43.510791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46548 (local address 127.3.73.195:34257)
0807 10:49:43.525551 (+ 14760us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:43.525564 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:43.525661 (+ 97us) server_negotiation.cc:407] Connection header received
0807 10:49:43.525788 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:43.525798 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:43.526123 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:43.527226 (+ 1103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:49:43.527915 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:43.530631 (+ 2716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:43.534029 (+ 3398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:43.536342 (+ 2313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:43.536504 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:43.536986 (+ 482us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:43.537377 (+ 391us) server_negotiation.cc:299] Negotiation successful
0807 10:49:43.537611 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14522,"thread_start_us":13521,"threads_started":1}
I20260807 10:49:43.544968 5682 tablet_service.cc:1467] Processing CreateTablet for tablet b56d404dbc234b46a49c4ebbdd972fe4 (DEFAULT_TABLE table=test-table [id=f03fc75a29e84902a4cad666426b547d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:49:43.547240 5682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b56d404dbc234b46a49c4ebbdd972fe4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:49:43.553200 5809 tablet_bootstrap.cc:492] T b56d404dbc234b46a49c4ebbdd972fe4 P ec508c2f08974d0b8c83a369c0d40616: Bootstrap starting.
I20260807 10:49:43.556490 5811 tablet_bootstrap.cc:492] T b56d404dbc234b46a49c4ebbdd972fe4 P 59c8fcaf1fd44e3d8fb950f658162cb0: Bootstrap starting.
I20260807 10:49:43.561100 5809 tablet_bootstrap.cc:654] T b56d404dbc234b46a49c4ebbdd972fe4 P ec508c2f08974d0b8c83a369c0d40616: Neither blocks nor log segments found. Creating new log.
I20260807 10:49:43.562741 5811 tablet_bootstrap.cc:654] T b56d404dbc234b46a49c4ebbdd972fe4 P 59c8fcaf1fd44e3d8fb950f658162cb0: Neither blocks nor log segments found. Creating new log.
I20260807 10:49:43.563653 5809 log.cc:824] T b56d404dbc234b46a49c4ebbdd972fe4 P ec508c2f08974d0b8c83a369c0d40616: Log is configured to *not* fsync() on all Append() calls
I20260807 10:49:43.565433 5811 log.cc:824] T b56d404dbc234b46a49c4ebbdd972fe4 P 59c8fcaf1fd44e3d8fb950f658162cb0: Log is configured to *not* fsync() on all Append() calls
I20260807 10:49:43.568503 5815 tablet_bootstrap.cc:492] T b56d404dbc234b46a49c4ebbdd972fe4 P 1c06edc993584b45a6fd20b3d6cb4ded: Bootstrap starting.
I20260807 10:49:43.569392 5809 tablet_bootstrap.cc:492] T b56d404dbc234b46a49c4ebbdd972fe4 P ec508c2f08974d0b8c83a369c0d40616: No bootstrap required, opened a new log
I20260807 10:49:43.569975 5809 ts_tablet_manager.cc:1397] T b56d404dbc234b46a49c4ebbdd972fe4 P ec508c2f08974d0b8c83a369c0d40616: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.009s
I20260807 10:49:43.574733 5815 tablet_bootstrap.cc:654] T b56d404dbc234b46a49c4ebbdd972fe4 P 1c06edc993584b45a6fd20b3d6cb4ded: Neither blocks nor log segments found. Creating new log.
I20260807 10:49:43.577545 5811 tablet_bootstrap.cc:492] T b56d404dbc234b46a49c4ebbdd972fe4 P 59c8fcaf1fd44e3d8fb950f658162cb0: No bootstrap required, opened a new log
I20260807 10:49:43.578003 5811 ts_tablet_manager.cc:1397] T b56d404dbc234b46a49c4ebbdd972fe4 P 59c8fcaf1fd44e3d8fb950f658162cb0: Time spent bootstrapping tablet: real 0.022s user 0.013s sys 0.007s
I20260807 10:49:43.577966 5815 log.cc:824] T b56d404dbc234b46a49c4ebbdd972fe4 P 1c06edc993584b45a6fd20b3d6cb4ded: Log is configured to *not* fsync() on all Append() calls
I20260807 10:49:43.582465 5815 tablet_bootstrap.cc:492] T b56d404dbc234b46a49c4ebbdd972fe4 P 1c06edc993584b45a6fd20b3d6cb4ded: No bootstrap required, opened a new log
I20260807 10:49:43.583034 5815 ts_tablet_manager.cc:1397] T b56d404dbc234b46a49c4ebbdd972fe4 P 1c06edc993584b45a6fd20b3d6cb4ded: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.006s
I20260807 10:49:43.586997 5809 raft_consensus.cc:348] T b56d404dbc234b46a49c4ebbdd972fe4 P ec508c2f08974d0b8c83a369c0d40616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } }
I20260807 10:49:43.587771 5809 raft_consensus.cc:374] T b56d404dbc234b46a49c4ebbdd972fe4 P ec508c2f08974d0b8c83a369c0d40616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:49:43.587942 5809 raft_consensus.cc:729] T b56d404dbc234b46a49c4ebbdd972fe4 P ec508c2f08974d0b8c83a369c0d40616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec508c2f08974d0b8c83a369c0d40616, State: Initialized, Role: FOLLOWER
I20260807 10:49:43.589007 5809 consensus_queue.cc:260] T b56d404dbc234b46a49c4ebbdd972fe4 P ec508c2f08974d0b8c83a369c0d40616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } }
I20260807 10:49:43.591780 5809 ts_tablet_manager.cc:1428] T b56d404dbc234b46a49c4ebbdd972fe4 P ec508c2f08974d0b8c83a369c0d40616: Time spent starting tablet: real 0.022s user 0.013s sys 0.008s
I20260807 10:49:43.594481 5811 raft_consensus.cc:348] T b56d404dbc234b46a49c4ebbdd972fe4 P 59c8fcaf1fd44e3d8fb950f658162cb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } }
I20260807 10:49:43.595017 5811 raft_consensus.cc:374] T b56d404dbc234b46a49c4ebbdd972fe4 P 59c8fcaf1fd44e3d8fb950f658162cb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:49:43.595230 5811 raft_consensus.cc:729] T b56d404dbc234b46a49c4ebbdd972fe4 P 59c8fcaf1fd44e3d8fb950f658162cb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59c8fcaf1fd44e3d8fb950f658162cb0, State: Initialized, Role: FOLLOWER
I20260807 10:49:43.595996 5811 consensus_queue.cc:260] T b56d404dbc234b46a49c4ebbdd972fe4 P 59c8fcaf1fd44e3d8fb950f658162cb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } }
I20260807 10:49:43.598979 5580 heartbeater.cc:503] Master 127.3.73.254:41707 was elected leader, sending a full tablet report...
I20260807 10:49:43.599819 5811 ts_tablet_manager.cc:1428] T b56d404dbc234b46a49c4ebbdd972fe4 P 59c8fcaf1fd44e3d8fb950f658162cb0: Time spent starting tablet: real 0.022s user 0.017s sys 0.004s
I20260807 10:49:43.602272 5815 raft_consensus.cc:348] T b56d404dbc234b46a49c4ebbdd972fe4 P 1c06edc993584b45a6fd20b3d6cb4ded [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } }
I20260807 10:49:43.602767 5815 raft_consensus.cc:374] T b56d404dbc234b46a49c4ebbdd972fe4 P 1c06edc993584b45a6fd20b3d6cb4ded [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:49:43.602955 5815 raft_consensus.cc:729] T b56d404dbc234b46a49c4ebbdd972fe4 P 1c06edc993584b45a6fd20b3d6cb4ded [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c06edc993584b45a6fd20b3d6cb4ded, State: Initialized, Role: FOLLOWER
I20260807 10:49:43.603869 5815 consensus_queue.cc:260] T b56d404dbc234b46a49c4ebbdd972fe4 P 1c06edc993584b45a6fd20b3d6cb4ded [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } }
I20260807 10:49:43.605584 5783 heartbeater.cc:503] Master 127.3.73.254:41707 was elected leader, sending a full tablet report...
I20260807 10:49:43.606606 5815 ts_tablet_manager.cc:1428] T b56d404dbc234b46a49c4ebbdd972fe4 P 1c06edc993584b45a6fd20b3d6cb4ded: Time spent starting tablet: real 0.023s user 0.012s sys 0.009s
W20260807 10:49:43.633237 5784 tablet.cc:2369] T b56d404dbc234b46a49c4ebbdd972fe4 P 1c06edc993584b45a6fd20b3d6cb4ded: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 10:49:43.689210 5372 tablet.cc:2369] T b56d404dbc234b46a49c4ebbdd972fe4 P ec508c2f08974d0b8c83a369c0d40616: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:49:43.731663 5820 raft_consensus.cc:482] T b56d404dbc234b46a49c4ebbdd972fe4 P ec508c2f08974d0b8c83a369c0d40616 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 10:49:43.732367 5820 raft_consensus.cc:504] T b56d404dbc234b46a49c4ebbdd972fe4 P ec508c2f08974d0b8c83a369c0d40616 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } }
I20260807 10:49:43.735671 5820 leader_election.cc:288] T b56d404dbc234b46a49c4ebbdd972fe4 P ec508c2f08974d0b8c83a369c0d40616 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59c8fcaf1fd44e3d8fb950f658162cb0 (127.3.73.194:34367), 1c06edc993584b45a6fd20b3d6cb4ded (127.3.73.195:34257)
I20260807 10:49:43.749518 5803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:43.736121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:40531 (local address 127.3.73.194:34367)
0807 10:49:43.737313 (+ 1192us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:43.737322 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:43.738280 (+ 958us) server_negotiation.cc:407] Connection header received
0807 10:49:43.738509 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:43.738517 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:43.738629 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:43.738790 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:49:43.740348 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:43.741981 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:43.746261 (+ 4280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:43.748169 (+ 1908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:43.748382 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:43.748474 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:43.749004 (+ 530us) server_negotiation.cc:299] Negotiation successful
0807 10:49:43.749157 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":898}
I20260807 10:49:43.749709 5807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:43.739265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:50425 (local address 127.3.73.195:34257)
0807 10:49:43.739552 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:43.739560 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:43.739582 (+ 22us) server_negotiation.cc:407] Connection header received
0807 10:49:43.739669 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:43.739676 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:43.739747 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:43.739848 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:49:43.741287 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:43.743115 (+ 1828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:43.746143 (+ 3028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:43.748501 (+ 2358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:43.748687 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:43.748813 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:43.749247 (+ 434us) server_negotiation.cc:299] Negotiation successful
0807 10:49:43.749391 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 10:49:43.750020 5830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:43.738260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:34257 (local address 127.3.73.193:50425)
0807 10:49:43.739298 (+ 1038us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:43.739345 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:43.739512 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:43.739873 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:43.739878 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:43.739907 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:49:43.741008 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:43.741036 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:43.743413 (+ 2377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:43.743418 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:43.745921 (+ 2503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:43.745935 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:43.748894 (+ 2959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:43.748901 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:43.749078 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:43.749103 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 10:49:43.749217 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 10:49:43.749355 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":822,"mutex_wait_us":24,"thread_start_us":1065,"threads_started":1}
I20260807 10:49:43.750331 5828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:43.736430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:34367 (local address 127.3.73.193:40531)
0807 10:49:43.738158 (+ 1728us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:43.738186 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:43.738344 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:43.738805 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:43.738811 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:43.738868 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:49:43.740123 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:43.740144 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:43.742121 (+ 1977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:43.742128 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:43.746044 (+ 3916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:43.746064 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:43.748486 (+ 2422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:43.748494 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:43.748802 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:43.748846 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 10:49:43.748971 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 10:49:43.749166 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1475,"mutex_wait_us":95,"thread_start_us":1003,"threads_started":1}
I20260807 10:49:43.751538 5714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b56d404dbc234b46a49c4ebbdd972fe4" candidate_uuid: "ec508c2f08974d0b8c83a369c0d40616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" is_pre_election: true
I20260807 10:49:43.751909 5510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b56d404dbc234b46a49c4ebbdd972fe4" candidate_uuid: "ec508c2f08974d0b8c83a369c0d40616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" is_pre_election: true
I20260807 10:49:43.752372 5714 raft_consensus.cc:2454] T b56d404dbc234b46a49c4ebbdd972fe4 P 1c06edc993584b45a6fd20b3d6cb4ded [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec508c2f08974d0b8c83a369c0d40616 in term 0.
I20260807 10:49:43.752806 5510 raft_consensus.cc:2454] T b56d404dbc234b46a49c4ebbdd972fe4 P 59c8fcaf1fd44e3d8fb950f658162cb0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec508c2f08974d0b8c83a369c0d40616 in term 0.
I20260807 10:49:43.753428 5187 leader_election.cc:302] T b56d404dbc234b46a49c4ebbdd972fe4 P ec508c2f08974d0b8c83a369c0d40616 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c06edc993584b45a6fd20b3d6cb4ded, ec508c2f08974d0b8c83a369c0d40616; no voters:
I20260807 10:49:43.754221 5820 raft_consensus.cc:2781] T b56d404dbc234b46a49c4ebbdd972fe4 P ec508c2f08974d0b8c83a369c0d40616 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 10:49:43.754575 5820 raft_consensus.cc:482] T b56d404dbc234b46a49c4ebbdd972fe4 P ec508c2f08974d0b8c83a369c0d40616 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 10:49:43.754848 5820 raft_consensus.cc:3037] T b56d404dbc234b46a49c4ebbdd972fe4 P ec508c2f08974d0b8c83a369c0d40616 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:49:43.758234 5820 raft_consensus.cc:504] T b56d404dbc234b46a49c4ebbdd972fe4 P ec508c2f08974d0b8c83a369c0d40616 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } }
I20260807 10:49:43.759003 5820 leader_election.cc:288] T b56d404dbc234b46a49c4ebbdd972fe4 P ec508c2f08974d0b8c83a369c0d40616 [CANDIDATE]: Term 1 election: Requested vote from peers 59c8fcaf1fd44e3d8fb950f658162cb0 (127.3.73.194:34367), 1c06edc993584b45a6fd20b3d6cb4ded (127.3.73.195:34257)
I20260807 10:49:43.759662 5510 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b56d404dbc234b46a49c4ebbdd972fe4" candidate_uuid: "ec508c2f08974d0b8c83a369c0d40616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0"
I20260807 10:49:43.759698 5714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b56d404dbc234b46a49c4ebbdd972fe4" candidate_uuid: "ec508c2f08974d0b8c83a369c0d40616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c06edc993584b45a6fd20b3d6cb4ded"
I20260807 10:49:43.759912 5714 raft_consensus.cc:3037] T b56d404dbc234b46a49c4ebbdd972fe4 P 1c06edc993584b45a6fd20b3d6cb4ded [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:49:43.759941 5510 raft_consensus.cc:3037] T b56d404dbc234b46a49c4ebbdd972fe4 P 59c8fcaf1fd44e3d8fb950f658162cb0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:49:43.762295 5714 raft_consensus.cc:2454] T b56d404dbc234b46a49c4ebbdd972fe4 P 1c06edc993584b45a6fd20b3d6cb4ded [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec508c2f08974d0b8c83a369c0d40616 in term 1.
I20260807 10:49:43.762825 5187 leader_election.cc:302] T b56d404dbc234b46a49c4ebbdd972fe4 P ec508c2f08974d0b8c83a369c0d40616 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c06edc993584b45a6fd20b3d6cb4ded, ec508c2f08974d0b8c83a369c0d40616; no voters:
I20260807 10:49:43.763126 5820 raft_consensus.cc:2781] T b56d404dbc234b46a49c4ebbdd972fe4 P ec508c2f08974d0b8c83a369c0d40616 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:49:43.763603 5510 raft_consensus.cc:2454] T b56d404dbc234b46a49c4ebbdd972fe4 P 59c8fcaf1fd44e3d8fb950f658162cb0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec508c2f08974d0b8c83a369c0d40616 in term 1.
I20260807 10:49:43.764003 5820 raft_consensus.cc:686] T b56d404dbc234b46a49c4ebbdd972fe4 P ec508c2f08974d0b8c83a369c0d40616 [term 1 LEADER]: Becoming Leader. State: Replica: ec508c2f08974d0b8c83a369c0d40616, State: Running, Role: LEADER
I20260807 10:49:43.764555 5820 consensus_queue.cc:237] T b56d404dbc234b46a49c4ebbdd972fe4 P ec508c2f08974d0b8c83a369c0d40616 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } }
I20260807 10:49:43.770061 5067 catalog_manager.cc:5495] T b56d404dbc234b46a49c4ebbdd972fe4 P ec508c2f08974d0b8c83a369c0d40616 reported cstate change: term changed from 0 to 1, leader changed from <none> to ec508c2f08974d0b8c83a369c0d40616 (127.3.73.193). New cstate: current_term: 1 leader_uuid: "ec508c2f08974d0b8c83a369c0d40616" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } health_report { overall_health: UNKNOWN } } }
W20260807 10:49:43.774358 5581 tablet.cc:2369] T b56d404dbc234b46a49c4ebbdd972fe4 P 59c8fcaf1fd44e3d8fb950f658162cb0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:49:43.823891 5805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:43.815856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50342 (local address 127.3.73.193:44329)
0807 10:49:43.816159 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:43.816165 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:43.816188 (+ 23us) server_negotiation.cc:407] Connection header received
0807 10:49:43.816580 (+ 392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:43.816585 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:43.816777 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:43.817166 (+ 389us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:49:43.817810 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:43.818916 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:43.820925 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:43.821524 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:43.821633 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:43.822460 (+ 827us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:49:43.823084 (+ 624us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:49:43.823205 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:43.823379 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 10:49:43.823454 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 10:49:43.836558 5714 raft_consensus.cc:1261] T b56d404dbc234b46a49c4ebbdd972fe4 P 1c06edc993584b45a6fd20b3d6cb4ded [term 1 FOLLOWER]: Refusing update from remote peer ec508c2f08974d0b8c83a369c0d40616: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:49:43.838233 5837 consensus_queue.cc:1035] T b56d404dbc234b46a49c4ebbdd972fe4 P ec508c2f08974d0b8c83a369c0d40616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:49:43.839027 5510 raft_consensus.cc:1261] T b56d404dbc234b46a49c4ebbdd972fe4 P 59c8fcaf1fd44e3d8fb950f658162cb0 [term 1 FOLLOWER]: Refusing update from remote peer ec508c2f08974d0b8c83a369c0d40616: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:49:43.840163 5837 consensus_queue.cc:1035] T b56d404dbc234b46a49c4ebbdd972fe4 P ec508c2f08974d0b8c83a369c0d40616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:49:43.888454 5064 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:49:43.889110 5064 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:49:43.895170 5064 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9a4a5d57a3a44ef6b4d184ffd17c4fd7: Sending DeleteTablet for 3 replicas of tablet b56d404dbc234b46a49c4ebbdd972fe4
I20260807 10:49:43.896965 5476 tablet_service.cc:1514] Processing DeleteTablet for tablet b56d404dbc234b46a49c4ebbdd972fe4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:49:43 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:49090
I20260807 10:49:43.897279 5263 tablet_service.cc:1514] Processing DeleteTablet for tablet b56d404dbc234b46a49c4ebbdd972fe4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:49:43 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:50326
I20260807 10:49:43.897449 5682 tablet_service.cc:1514] Processing DeleteTablet for tablet b56d404dbc234b46a49c4ebbdd972fe4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:49:43 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:46548
I20260807 10:49:43.899580 5858 tablet_replica.cc:330] T b56d404dbc234b46a49c4ebbdd972fe4 P ec508c2f08974d0b8c83a369c0d40616: stopping tablet replica
I20260807 10:49:43.900509 5858 raft_consensus.cc:2229] T b56d404dbc234b46a49c4ebbdd972fe4 P ec508c2f08974d0b8c83a369c0d40616 [term 1 LEADER]: Raft consensus shutting down.
I20260807 10:49:43.900187 5064 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53116:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 10:49:43.901803 5858 raft_consensus.cc:2258] T b56d404dbc234b46a49c4ebbdd972fe4 P ec508c2f08974d0b8c83a369c0d40616 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 10:49:43.902417 5064 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 10:49:43.905951 5859 tablet_replica.cc:330] T b56d404dbc234b46a49c4ebbdd972fe4 P 1c06edc993584b45a6fd20b3d6cb4ded: stopping tablet replica
I20260807 10:49:43.906562 5859 raft_consensus.cc:2229] T b56d404dbc234b46a49c4ebbdd972fe4 P 1c06edc993584b45a6fd20b3d6cb4ded [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:49:43.906862 5857 tablet_replica.cc:330] T b56d404dbc234b46a49c4ebbdd972fe4 P 59c8fcaf1fd44e3d8fb950f658162cb0: stopping tablet replica
I20260807 10:49:43.907461 5857 raft_consensus.cc:2229] T b56d404dbc234b46a49c4ebbdd972fe4 P 59c8fcaf1fd44e3d8fb950f658162cb0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:49:43.908226 5859 raft_consensus.cc:2258] T b56d404dbc234b46a49c4ebbdd972fe4 P 1c06edc993584b45a6fd20b3d6cb4ded [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:49:43.908304 5857 raft_consensus.cc:2258] T b56d404dbc234b46a49c4ebbdd972fe4 P 59c8fcaf1fd44e3d8fb950f658162cb0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:49:43.910900 5858 ts_tablet_manager.cc:1905] T b56d404dbc234b46a49c4ebbdd972fe4 P ec508c2f08974d0b8c83a369c0d40616: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:49:43.911381 5859 ts_tablet_manager.cc:1905] T b56d404dbc234b46a49c4ebbdd972fe4 P 1c06edc993584b45a6fd20b3d6cb4ded: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:49:43.912667 5857 ts_tablet_manager.cc:1905] T b56d404dbc234b46a49c4ebbdd972fe4 P 59c8fcaf1fd44e3d8fb950f658162cb0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:49:43.918466 5263 tablet_service.cc:1467] Processing CreateTablet for tablet bab91472e2694999900fed238ee19ead (DEFAULT_TABLE table=test-table [id=1c81e0f9210f45e2b37a5a042dd1895c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:49:43.919273 5263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bab91472e2694999900fed238ee19ead. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:49:43.921244 5682 tablet_service.cc:1467] Processing CreateTablet for tablet bab91472e2694999900fed238ee19ead (DEFAULT_TABLE table=test-table [id=1c81e0f9210f45e2b37a5a042dd1895c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:49:43.921492 5859 ts_tablet_manager.cc:1918] T b56d404dbc234b46a49c4ebbdd972fe4 P 1c06edc993584b45a6fd20b3d6cb4ded: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:49:43.921802 5859 log.cc:1196] T b56d404dbc234b46a49c4ebbdd972fe4 P 1c06edc993584b45a6fd20b3d6cb4ded: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-2/wal/wals/b56d404dbc234b46a49c4ebbdd972fe4
I20260807 10:49:43.922006 5682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bab91472e2694999900fed238ee19ead. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:49:43.922633 5859 ts_tablet_manager.cc:1939] T b56d404dbc234b46a49c4ebbdd972fe4 P 1c06edc993584b45a6fd20b3d6cb4ded: Deleting consensus metadata
I20260807 10:49:43.924404 5476 tablet_service.cc:1467] Processing CreateTablet for tablet bab91472e2694999900fed238ee19ead (DEFAULT_TABLE table=test-table [id=1c81e0f9210f45e2b37a5a042dd1895c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:49:43.924638 5857 ts_tablet_manager.cc:1918] T b56d404dbc234b46a49c4ebbdd972fe4 P 59c8fcaf1fd44e3d8fb950f658162cb0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:49:43.925213 5476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bab91472e2694999900fed238ee19ead. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:49:43.925395 5857 log.cc:1196] T b56d404dbc234b46a49c4ebbdd972fe4 P 59c8fcaf1fd44e3d8fb950f658162cb0: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-1/wal/wals/b56d404dbc234b46a49c4ebbdd972fe4
I20260807 10:49:43.926518 5857 ts_tablet_manager.cc:1939] T b56d404dbc234b46a49c4ebbdd972fe4 P 59c8fcaf1fd44e3d8fb950f658162cb0: Deleting consensus metadata
I20260807 10:49:43.929718 5045 catalog_manager.cc:4841] TS 59c8fcaf1fd44e3d8fb950f658162cb0 (127.3.73.194:34367): tablet b56d404dbc234b46a49c4ebbdd972fe4 (table test-table [id=f03fc75a29e84902a4cad666426b547d]) successfully deleted
I20260807 10:49:43.932739 5815 tablet_bootstrap.cc:492] T bab91472e2694999900fed238ee19ead P 1c06edc993584b45a6fd20b3d6cb4ded: Bootstrap starting.
I20260807 10:49:43.932711 5045 catalog_manager.cc:4841] TS 1c06edc993584b45a6fd20b3d6cb4ded (127.3.73.195:34257): tablet b56d404dbc234b46a49c4ebbdd972fe4 (table test-table [id=f03fc75a29e84902a4cad666426b547d]) successfully deleted
I20260807 10:49:43.933431 5858 ts_tablet_manager.cc:1918] T b56d404dbc234b46a49c4ebbdd972fe4 P ec508c2f08974d0b8c83a369c0d40616: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:49:43.933773 5858 log.cc:1196] T b56d404dbc234b46a49c4ebbdd972fe4 P ec508c2f08974d0b8c83a369c0d40616: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-0/wal/wals/b56d404dbc234b46a49c4ebbdd972fe4
I20260807 10:49:43.935901 5815 tablet_bootstrap.cc:654] T bab91472e2694999900fed238ee19ead P 1c06edc993584b45a6fd20b3d6cb4ded: Neither blocks nor log segments found. Creating new log.
I20260807 10:49:43.936415 5811 tablet_bootstrap.cc:492] T bab91472e2694999900fed238ee19ead P 59c8fcaf1fd44e3d8fb950f658162cb0: Bootstrap starting.
I20260807 10:49:43.936661 5858 ts_tablet_manager.cc:1939] T b56d404dbc234b46a49c4ebbdd972fe4 P ec508c2f08974d0b8c83a369c0d40616: Deleting consensus metadata
I20260807 10:49:43.939715 5046 catalog_manager.cc:4841] TS ec508c2f08974d0b8c83a369c0d40616 (127.3.73.193:44329): tablet b56d404dbc234b46a49c4ebbdd972fe4 (table test-table [id=f03fc75a29e84902a4cad666426b547d]) successfully deleted
I20260807 10:49:43.941784 5811 tablet_bootstrap.cc:654] T bab91472e2694999900fed238ee19ead P 59c8fcaf1fd44e3d8fb950f658162cb0: Neither blocks nor log segments found. Creating new log.
I20260807 10:49:43.942896 5815 tablet_bootstrap.cc:492] T bab91472e2694999900fed238ee19ead P 1c06edc993584b45a6fd20b3d6cb4ded: No bootstrap required, opened a new log
I20260807 10:49:43.943171 5815 ts_tablet_manager.cc:1397] T bab91472e2694999900fed238ee19ead P 1c06edc993584b45a6fd20b3d6cb4ded: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.003s
I20260807 10:49:43.944917 5815 raft_consensus.cc:348] T bab91472e2694999900fed238ee19ead P 1c06edc993584b45a6fd20b3d6cb4ded [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } }
I20260807 10:49:43.945248 5815 raft_consensus.cc:374] T bab91472e2694999900fed238ee19ead P 1c06edc993584b45a6fd20b3d6cb4ded [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:49:43.945380 5815 raft_consensus.cc:729] T bab91472e2694999900fed238ee19ead P 1c06edc993584b45a6fd20b3d6cb4ded [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c06edc993584b45a6fd20b3d6cb4ded, State: Initialized, Role: FOLLOWER
I20260807 10:49:43.945783 5815 consensus_queue.cc:260] T bab91472e2694999900fed238ee19ead P 1c06edc993584b45a6fd20b3d6cb4ded [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } }
I20260807 10:49:43.946264 5809 tablet_bootstrap.cc:492] T bab91472e2694999900fed238ee19ead P ec508c2f08974d0b8c83a369c0d40616: Bootstrap starting.
I20260807 10:49:43.946803 5815 ts_tablet_manager.cc:1428] T bab91472e2694999900fed238ee19ead P 1c06edc993584b45a6fd20b3d6cb4ded: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 10:49:43.949235 5809 tablet_bootstrap.cc:654] T bab91472e2694999900fed238ee19ead P ec508c2f08974d0b8c83a369c0d40616: Neither blocks nor log segments found. Creating new log.
I20260807 10:49:43.951227 5811 tablet_bootstrap.cc:492] T bab91472e2694999900fed238ee19ead P 59c8fcaf1fd44e3d8fb950f658162cb0: No bootstrap required, opened a new log
I20260807 10:49:43.951550 5811 ts_tablet_manager.cc:1397] T bab91472e2694999900fed238ee19ead P 59c8fcaf1fd44e3d8fb950f658162cb0: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.003s
I20260807 10:49:43.953255 5811 raft_consensus.cc:348] T bab91472e2694999900fed238ee19ead P 59c8fcaf1fd44e3d8fb950f658162cb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } }
I20260807 10:49:43.953670 5811 raft_consensus.cc:374] T bab91472e2694999900fed238ee19ead P 59c8fcaf1fd44e3d8fb950f658162cb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:49:43.953866 5811 raft_consensus.cc:729] T bab91472e2694999900fed238ee19ead P 59c8fcaf1fd44e3d8fb950f658162cb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59c8fcaf1fd44e3d8fb950f658162cb0, State: Initialized, Role: FOLLOWER
I20260807 10:49:43.954319 5811 consensus_queue.cc:260] T bab91472e2694999900fed238ee19ead P 59c8fcaf1fd44e3d8fb950f658162cb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } }
I20260807 10:49:43.955336 5811 ts_tablet_manager.cc:1428] T bab91472e2694999900fed238ee19ead P 59c8fcaf1fd44e3d8fb950f658162cb0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 10:49:43.960391 5809 tablet_bootstrap.cc:492] T bab91472e2694999900fed238ee19ead P ec508c2f08974d0b8c83a369c0d40616: No bootstrap required, opened a new log
I20260807 10:49:43.960644 5809 ts_tablet_manager.cc:1397] T bab91472e2694999900fed238ee19ead P ec508c2f08974d0b8c83a369c0d40616: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.004s
I20260807 10:49:43.962213 5809 raft_consensus.cc:348] T bab91472e2694999900fed238ee19ead P ec508c2f08974d0b8c83a369c0d40616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } }
I20260807 10:49:43.962596 5809 raft_consensus.cc:374] T bab91472e2694999900fed238ee19ead P ec508c2f08974d0b8c83a369c0d40616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:49:43.962846 5809 raft_consensus.cc:729] T bab91472e2694999900fed238ee19ead P ec508c2f08974d0b8c83a369c0d40616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec508c2f08974d0b8c83a369c0d40616, State: Initialized, Role: FOLLOWER
I20260807 10:49:43.963387 5809 consensus_queue.cc:260] T bab91472e2694999900fed238ee19ead P ec508c2f08974d0b8c83a369c0d40616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } }
I20260807 10:49:43.964412 5809 ts_tablet_manager.cc:1428] T bab91472e2694999900fed238ee19ead P ec508c2f08974d0b8c83a369c0d40616: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 10:49:44.117651 5821 raft_consensus.cc:482] T bab91472e2694999900fed238ee19ead P 59c8fcaf1fd44e3d8fb950f658162cb0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 10:49:44.118212 5821 raft_consensus.cc:504] T bab91472e2694999900fed238ee19ead P 59c8fcaf1fd44e3d8fb950f658162cb0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } }
I20260807 10:49:44.121567 5821 leader_election.cc:288] T bab91472e2694999900fed238ee19ead P 59c8fcaf1fd44e3d8fb950f658162cb0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec508c2f08974d0b8c83a369c0d40616 (127.3.73.193:44329), 1c06edc993584b45a6fd20b3d6cb4ded (127.3.73.195:34257)
I20260807 10:49:44.141883 5807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:44.122627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:39819 (local address 127.3.73.195:34257)
0807 10:49:44.122900 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:44.122909 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:44.134359 (+ 11450us) server_negotiation.cc:407] Connection header received
0807 10:49:44.134750 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:44.134758 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:44.134861 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:44.135000 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:49:44.135960 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:44.137041 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:44.139101 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:44.140524 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:44.140707 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:44.140817 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:44.141429 (+ 612us) server_negotiation.cc:299] Negotiation successful
0807 10:49:44.141578 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 10:49:44.142123 5805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:44.128961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:34381 (local address 127.3.73.193:44329)
0807 10:49:44.129220 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:44.129228 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:44.133789 (+ 4561us) server_negotiation.cc:407] Connection header received
0807 10:49:44.134167 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:44.134174 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:44.134258 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:44.134363 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:49:44.135358 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:44.136504 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:44.139455 (+ 2951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:44.140870 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:44.141075 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:44.141157 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:44.141582 (+ 425us) server_negotiation.cc:299] Negotiation successful
0807 10:49:44.141726 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260807 10:49:44.142386 5866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:44.122021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:44329 (local address 127.3.73.194:34381)
0807 10:49:44.133557 (+ 11536us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:44.133587 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:44.133988 (+ 401us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:44.134390 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:44.134395 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:44.134424 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:49:44.135072 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:44.135088 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:44.136637 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:44.136643 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:44.139176 (+ 2533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:44.139192 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:44.141139 (+ 1947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:44.141146 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:44.141393 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:44.141431 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 10:49:44.141544 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 10:49:44.141683 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11318,"mutex_wait_us":1,"spinlock_wait_cycles":157952,"thread_start_us":10716,"threads_started":1}
I20260807 10:49:44.142800 5867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:44.123224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:34257 (local address 127.3.73.194:39819)
0807 10:49:44.134229 (+ 11005us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:44.134250 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:44.134533 (+ 283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:44.135044 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:44.135052 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:44.135081 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:49:44.135777 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:44.135790 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:44.137173 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:44.137180 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:44.138945 (+ 1765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:44.138959 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:44.140820 (+ 1861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:44.140829 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:44.141067 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:44.141105 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 10:49:44.142220 (+ 1115us) client_negotiation.cc:240] Negotiation successful
0807 10:49:44.142382 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10761,"thread_start_us":10488,"threads_started":1}
I20260807 10:49:44.143805 5714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bab91472e2694999900fed238ee19ead" candidate_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" is_pre_election: true
I20260807 10:49:44.144105 5714 raft_consensus.cc:2454] T bab91472e2694999900fed238ee19ead P 1c06edc993584b45a6fd20b3d6cb4ded [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59c8fcaf1fd44e3d8fb950f658162cb0 in term 0.
I20260807 10:49:44.143874 5304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bab91472e2694999900fed238ee19ead" candidate_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec508c2f08974d0b8c83a369c0d40616" is_pre_election: true
I20260807 10:49:44.144454 5304 raft_consensus.cc:2454] T bab91472e2694999900fed238ee19ead P ec508c2f08974d0b8c83a369c0d40616 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59c8fcaf1fd44e3d8fb950f658162cb0 in term 0.
I20260807 10:49:44.144646 5403 leader_election.cc:302] T bab91472e2694999900fed238ee19ead P 59c8fcaf1fd44e3d8fb950f658162cb0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c06edc993584b45a6fd20b3d6cb4ded, 59c8fcaf1fd44e3d8fb950f658162cb0; no voters:
I20260807 10:49:44.145246 5821 raft_consensus.cc:2781] T bab91472e2694999900fed238ee19ead P 59c8fcaf1fd44e3d8fb950f658162cb0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 10:49:44.145408 5821 raft_consensus.cc:482] T bab91472e2694999900fed238ee19ead P 59c8fcaf1fd44e3d8fb950f658162cb0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 10:49:44.145526 5821 raft_consensus.cc:3037] T bab91472e2694999900fed238ee19ead P 59c8fcaf1fd44e3d8fb950f658162cb0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:49:44.147769 5821 raft_consensus.cc:504] T bab91472e2694999900fed238ee19ead P 59c8fcaf1fd44e3d8fb950f658162cb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } }
I20260807 10:49:44.148480 5821 leader_election.cc:288] T bab91472e2694999900fed238ee19ead P 59c8fcaf1fd44e3d8fb950f658162cb0 [CANDIDATE]: Term 1 election: Requested vote from peers ec508c2f08974d0b8c83a369c0d40616 (127.3.73.193:44329), 1c06edc993584b45a6fd20b3d6cb4ded (127.3.73.195:34257)
I20260807 10:49:44.149000 5304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bab91472e2694999900fed238ee19ead" candidate_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec508c2f08974d0b8c83a369c0d40616"
I20260807 10:49:44.149262 5304 raft_consensus.cc:3037] T bab91472e2694999900fed238ee19ead P ec508c2f08974d0b8c83a369c0d40616 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:49:44.149392 5714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bab91472e2694999900fed238ee19ead" candidate_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c06edc993584b45a6fd20b3d6cb4ded"
I20260807 10:49:44.149739 5714 raft_consensus.cc:3037] T bab91472e2694999900fed238ee19ead P 1c06edc993584b45a6fd20b3d6cb4ded [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:49:44.152022 5714 raft_consensus.cc:2454] T bab91472e2694999900fed238ee19ead P 1c06edc993584b45a6fd20b3d6cb4ded [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59c8fcaf1fd44e3d8fb950f658162cb0 in term 1.
I20260807 10:49:44.152488 5403 leader_election.cc:302] T bab91472e2694999900fed238ee19ead P 59c8fcaf1fd44e3d8fb950f658162cb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c06edc993584b45a6fd20b3d6cb4ded, 59c8fcaf1fd44e3d8fb950f658162cb0; no voters:
I20260807 10:49:44.152700 5304 raft_consensus.cc:2454] T bab91472e2694999900fed238ee19ead P ec508c2f08974d0b8c83a369c0d40616 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59c8fcaf1fd44e3d8fb950f658162cb0 in term 1.
I20260807 10:49:44.152819 5821 raft_consensus.cc:2781] T bab91472e2694999900fed238ee19ead P 59c8fcaf1fd44e3d8fb950f658162cb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:49:44.153569 5821 raft_consensus.cc:686] T bab91472e2694999900fed238ee19ead P 59c8fcaf1fd44e3d8fb950f658162cb0 [term 1 LEADER]: Becoming Leader. State: Replica: 59c8fcaf1fd44e3d8fb950f658162cb0, State: Running, Role: LEADER
I20260807 10:49:44.154033 5821 consensus_queue.cc:237] T bab91472e2694999900fed238ee19ead P 59c8fcaf1fd44e3d8fb950f658162cb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } }
I20260807 10:49:44.157914 5064 catalog_manager.cc:5495] T bab91472e2694999900fed238ee19ead P 59c8fcaf1fd44e3d8fb950f658162cb0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 59c8fcaf1fd44e3d8fb950f658162cb0 (127.3.73.194). New cstate: current_term: 1 leader_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } health_report { overall_health: UNKNOWN } } }
Aug 07 10:49:44 dist-test-slave-x5fp krb5kdc[4996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099781, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:49:44.244328 5878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:44.219733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53122 (local address 127.3.73.254:41707)
0807 10:49:44.221267 (+ 1534us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:44.221275 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:44.221981 (+ 706us) server_negotiation.cc:407] Connection header received
0807 10:49:44.222138 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:44.222145 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:44.222231 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:44.222382 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:44.224341 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:44.226454 (+ 2113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:44.228944 (+ 2490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:44.230231 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:44.230419 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:44.235265 (+ 4846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:44.235275 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:44.235281 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:44.235326 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:44.239928 (+ 4602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:44.241232 (+ 1304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:44.241237 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:44.241280 (+ 43us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:44.241501 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:44.242082 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:44.242087 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:44.242111 (+ 24us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:44.242573 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:44.242781 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:44.243381 (+ 600us) server_negotiation.cc:299] Negotiation successful
0807 10:49:44.243693 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1296,"thread_start_us":1008,"threads_started":1}
I20260807 10:49:44.244392 5879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:44.219734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41707 (local address 127.0.0.1:53122)
0807 10:49:44.221783 (+ 2049us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:44.221801 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:44.221960 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:44.222408 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:44.222411 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:44.223561 (+ 1150us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:49:44.224065 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:44.224078 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:44.226694 (+ 2616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:44.226698 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:44.228630 (+ 1932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:44.228645 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:44.230467 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:44.230472 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:44.230694 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:44.231614 (+ 920us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:49:44.231634 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:49:44.234989 (+ 3355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:49:44.240123 (+ 5134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:44.240146 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:44.240154 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:44.241102 (+ 948us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:44.241650 (+ 548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:44.241654 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:44.241658 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:44.241963 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:44.242766 (+ 803us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:49:44.242775 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:49:44.243019 (+ 244us) client_negotiation.cc:769] Sending connection context
0807 10:49:44.243256 (+ 237us) client_negotiation.cc:240] Negotiation successful
0807 10:49:44.243859 (+ 603us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1883,"mutex_wait_us":222,"thread_start_us":1717,"threads_started":1}
I20260807 10:49:44.251760 5064 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:53122:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 10:49:44.253113 5064 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 10:49:44.261922 5263 tablet_service.cc:1467] Processing CreateTablet for tablet eea67a9585694b11a0fb56c3bc4510ca (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5e12cc919ab24c85a2b208f56b5f2a72]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 10:49:44.262284 5682 tablet_service.cc:1467] Processing CreateTablet for tablet eea67a9585694b11a0fb56c3bc4510ca (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5e12cc919ab24c85a2b208f56b5f2a72]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 10:49:44.262362 5476 tablet_service.cc:1467] Processing CreateTablet for tablet eea67a9585694b11a0fb56c3bc4510ca (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5e12cc919ab24c85a2b208f56b5f2a72]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 10:49:44.262681 5263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eea67a9585694b11a0fb56c3bc4510ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:49:44.263085 5476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eea67a9585694b11a0fb56c3bc4510ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:49:44.263180 5682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eea67a9585694b11a0fb56c3bc4510ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:49:44.273316 5811 tablet_bootstrap.cc:492] T eea67a9585694b11a0fb56c3bc4510ca P 59c8fcaf1fd44e3d8fb950f658162cb0: Bootstrap starting.
I20260807 10:49:44.273316 5809 tablet_bootstrap.cc:492] T eea67a9585694b11a0fb56c3bc4510ca P ec508c2f08974d0b8c83a369c0d40616: Bootstrap starting.
I20260807 10:49:44.276764 5811 tablet_bootstrap.cc:654] T eea67a9585694b11a0fb56c3bc4510ca P 59c8fcaf1fd44e3d8fb950f658162cb0: Neither blocks nor log segments found. Creating new log.
I20260807 10:49:44.276993 5809 tablet_bootstrap.cc:654] T eea67a9585694b11a0fb56c3bc4510ca P ec508c2f08974d0b8c83a369c0d40616: Neither blocks nor log segments found. Creating new log.
I20260807 10:49:44.280140 5815 tablet_bootstrap.cc:492] T eea67a9585694b11a0fb56c3bc4510ca P 1c06edc993584b45a6fd20b3d6cb4ded: Bootstrap starting.
I20260807 10:49:44.283294 5815 tablet_bootstrap.cc:654] T eea67a9585694b11a0fb56c3bc4510ca P 1c06edc993584b45a6fd20b3d6cb4ded: Neither blocks nor log segments found. Creating new log.
I20260807 10:49:44.288178 5811 tablet_bootstrap.cc:492] T eea67a9585694b11a0fb56c3bc4510ca P 59c8fcaf1fd44e3d8fb950f658162cb0: No bootstrap required, opened a new log
I20260807 10:49:44.288560 5811 ts_tablet_manager.cc:1397] T eea67a9585694b11a0fb56c3bc4510ca P 59c8fcaf1fd44e3d8fb950f658162cb0: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.009s
I20260807 10:49:44.289312 5809 tablet_bootstrap.cc:492] T eea67a9585694b11a0fb56c3bc4510ca P ec508c2f08974d0b8c83a369c0d40616: No bootstrap required, opened a new log
I20260807 10:49:44.289662 5809 ts_tablet_manager.cc:1397] T eea67a9585694b11a0fb56c3bc4510ca P ec508c2f08974d0b8c83a369c0d40616: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.008s
I20260807 10:49:44.290005 5811 raft_consensus.cc:348] T eea67a9585694b11a0fb56c3bc4510ca P 59c8fcaf1fd44e3d8fb950f658162cb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } }
I20260807 10:49:44.290398 5811 raft_consensus.cc:374] T eea67a9585694b11a0fb56c3bc4510ca P 59c8fcaf1fd44e3d8fb950f658162cb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:49:44.290520 5811 raft_consensus.cc:729] T eea67a9585694b11a0fb56c3bc4510ca P 59c8fcaf1fd44e3d8fb950f658162cb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59c8fcaf1fd44e3d8fb950f658162cb0, State: Initialized, Role: FOLLOWER
I20260807 10:49:44.290931 5809 raft_consensus.cc:348] T eea67a9585694b11a0fb56c3bc4510ca P ec508c2f08974d0b8c83a369c0d40616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } }
I20260807 10:49:44.291043 5811 consensus_queue.cc:260] T eea67a9585694b11a0fb56c3bc4510ca P 59c8fcaf1fd44e3d8fb950f658162cb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } }
I20260807 10:49:44.291378 5809 raft_consensus.cc:374] T eea67a9585694b11a0fb56c3bc4510ca P ec508c2f08974d0b8c83a369c0d40616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:49:44.291558 5809 raft_consensus.cc:729] T eea67a9585694b11a0fb56c3bc4510ca P ec508c2f08974d0b8c83a369c0d40616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec508c2f08974d0b8c83a369c0d40616, State: Initialized, Role: FOLLOWER
I20260807 10:49:44.291772 5821 tablet_replica.cc:439] T eea67a9585694b11a0fb56c3bc4510ca P 59c8fcaf1fd44e3d8fb950f658162cb0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } } }
I20260807 10:49:44.292232 5821 tablet_replica.cc:442] T eea67a9585694b11a0fb56c3bc4510ca P 59c8fcaf1fd44e3d8fb950f658162cb0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 10:49:44.292009 5809 consensus_queue.cc:260] T eea67a9585694b11a0fb56c3bc4510ca P ec508c2f08974d0b8c83a369c0d40616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } }
I20260807 10:49:44.293016 5809 ts_tablet_manager.cc:1428] T eea67a9585694b11a0fb56c3bc4510ca P ec508c2f08974d0b8c83a369c0d40616: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260807 10:49:44.292819 5837 tablet_replica.cc:439] T eea67a9585694b11a0fb56c3bc4510ca P ec508c2f08974d0b8c83a369c0d40616: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } } }
I20260807 10:49:44.293298 5837 tablet_replica.cc:442] T eea67a9585694b11a0fb56c3bc4510ca P ec508c2f08974d0b8c83a369c0d40616: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 10:49:44.296553 5811 ts_tablet_manager.cc:1428] T eea67a9585694b11a0fb56c3bc4510ca P 59c8fcaf1fd44e3d8fb950f658162cb0: Time spent starting tablet: real 0.008s user 0.004s sys 0.000s
I20260807 10:49:44.301419 5815 tablet_bootstrap.cc:492] T eea67a9585694b11a0fb56c3bc4510ca P 1c06edc993584b45a6fd20b3d6cb4ded: No bootstrap required, opened a new log
I20260807 10:49:44.301745 5815 ts_tablet_manager.cc:1397] T eea67a9585694b11a0fb56c3bc4510ca P 1c06edc993584b45a6fd20b3d6cb4ded: Time spent bootstrapping tablet: real 0.022s user 0.009s sys 0.004s
I20260807 10:49:44.303265 5815 raft_consensus.cc:348] T eea67a9585694b11a0fb56c3bc4510ca P 1c06edc993584b45a6fd20b3d6cb4ded [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } }
I20260807 10:49:44.303665 5815 raft_consensus.cc:374] T eea67a9585694b11a0fb56c3bc4510ca P 1c06edc993584b45a6fd20b3d6cb4ded [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:49:44.303826 5815 raft_consensus.cc:729] T eea67a9585694b11a0fb56c3bc4510ca P 1c06edc993584b45a6fd20b3d6cb4ded [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c06edc993584b45a6fd20b3d6cb4ded, State: Initialized, Role: FOLLOWER
I20260807 10:49:44.304214 5815 consensus_queue.cc:260] T eea67a9585694b11a0fb56c3bc4510ca P 1c06edc993584b45a6fd20b3d6cb4ded [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } }
I20260807 10:49:44.305202 5815 ts_tablet_manager.cc:1428] T eea67a9585694b11a0fb56c3bc4510ca P 1c06edc993584b45a6fd20b3d6cb4ded: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260807 10:49:44.305135 5824 tablet_replica.cc:439] T eea67a9585694b11a0fb56c3bc4510ca P 1c06edc993584b45a6fd20b3d6cb4ded: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } } }
I20260807 10:49:44.305807 5824 tablet_replica.cc:442] T eea67a9585694b11a0fb56c3bc4510ca P 1c06edc993584b45a6fd20b3d6cb4ded: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 10:49:44.352388 5821 raft_consensus.cc:482] T eea67a9585694b11a0fb56c3bc4510ca P 59c8fcaf1fd44e3d8fb950f658162cb0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 10:49:44.352864 5821 raft_consensus.cc:504] T eea67a9585694b11a0fb56c3bc4510ca P 59c8fcaf1fd44e3d8fb950f658162cb0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } }
I20260807 10:49:44.353922 5821 leader_election.cc:288] T eea67a9585694b11a0fb56c3bc4510ca P 59c8fcaf1fd44e3d8fb950f658162cb0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec508c2f08974d0b8c83a369c0d40616 (127.3.73.193:44329), 1c06edc993584b45a6fd20b3d6cb4ded (127.3.73.195:34257)
I20260807 10:49:44.354565 5714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eea67a9585694b11a0fb56c3bc4510ca" candidate_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" is_pre_election: true
I20260807 10:49:44.354683 5304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eea67a9585694b11a0fb56c3bc4510ca" candidate_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec508c2f08974d0b8c83a369c0d40616" is_pre_election: true
I20260807 10:49:44.354918 5714 raft_consensus.cc:2454] T eea67a9585694b11a0fb56c3bc4510ca P 1c06edc993584b45a6fd20b3d6cb4ded [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59c8fcaf1fd44e3d8fb950f658162cb0 in term 0.
I20260807 10:49:44.355082 5304 raft_consensus.cc:2454] T eea67a9585694b11a0fb56c3bc4510ca P ec508c2f08974d0b8c83a369c0d40616 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59c8fcaf1fd44e3d8fb950f658162cb0 in term 0.
I20260807 10:49:44.355520 5403 leader_election.cc:302] T eea67a9585694b11a0fb56c3bc4510ca P 59c8fcaf1fd44e3d8fb950f658162cb0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c06edc993584b45a6fd20b3d6cb4ded, 59c8fcaf1fd44e3d8fb950f658162cb0; no voters:
I20260807 10:49:44.355939 5821 raft_consensus.cc:2781] T eea67a9585694b11a0fb56c3bc4510ca P 59c8fcaf1fd44e3d8fb950f658162cb0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 10:49:44.356101 5821 raft_consensus.cc:482] T eea67a9585694b11a0fb56c3bc4510ca P 59c8fcaf1fd44e3d8fb950f658162cb0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 10:49:44.356431 5821 raft_consensus.cc:3037] T eea67a9585694b11a0fb56c3bc4510ca P 59c8fcaf1fd44e3d8fb950f658162cb0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:49:44.361256 5821 raft_consensus.cc:504] T eea67a9585694b11a0fb56c3bc4510ca P 59c8fcaf1fd44e3d8fb950f658162cb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } }
I20260807 10:49:44.362318 5821 leader_election.cc:288] T eea67a9585694b11a0fb56c3bc4510ca P 59c8fcaf1fd44e3d8fb950f658162cb0 [CANDIDATE]: Term 1 election: Requested vote from peers ec508c2f08974d0b8c83a369c0d40616 (127.3.73.193:44329), 1c06edc993584b45a6fd20b3d6cb4ded (127.3.73.195:34257)
I20260807 10:49:44.362859 5304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eea67a9585694b11a0fb56c3bc4510ca" candidate_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec508c2f08974d0b8c83a369c0d40616"
I20260807 10:49:44.362975 5714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eea67a9585694b11a0fb56c3bc4510ca" candidate_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c06edc993584b45a6fd20b3d6cb4ded"
I20260807 10:49:44.363154 5304 raft_consensus.cc:3037] T eea67a9585694b11a0fb56c3bc4510ca P ec508c2f08974d0b8c83a369c0d40616 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:49:44.363440 5714 raft_consensus.cc:3037] T eea67a9585694b11a0fb56c3bc4510ca P 1c06edc993584b45a6fd20b3d6cb4ded [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:49:44.366449 5304 raft_consensus.cc:2454] T eea67a9585694b11a0fb56c3bc4510ca P ec508c2f08974d0b8c83a369c0d40616 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59c8fcaf1fd44e3d8fb950f658162cb0 in term 1.
I20260807 10:49:44.367105 5405 leader_election.cc:302] T eea67a9585694b11a0fb56c3bc4510ca P 59c8fcaf1fd44e3d8fb950f658162cb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59c8fcaf1fd44e3d8fb950f658162cb0, ec508c2f08974d0b8c83a369c0d40616; no voters:
I20260807 10:49:44.367511 5821 raft_consensus.cc:2781] T eea67a9585694b11a0fb56c3bc4510ca P 59c8fcaf1fd44e3d8fb950f658162cb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:49:44.367714 5821 raft_consensus.cc:686] T eea67a9585694b11a0fb56c3bc4510ca P 59c8fcaf1fd44e3d8fb950f658162cb0 [term 1 LEADER]: Becoming Leader. State: Replica: 59c8fcaf1fd44e3d8fb950f658162cb0, State: Running, Role: LEADER
I20260807 10:49:44.367671 5714 raft_consensus.cc:2454] T eea67a9585694b11a0fb56c3bc4510ca P 1c06edc993584b45a6fd20b3d6cb4ded [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59c8fcaf1fd44e3d8fb950f658162cb0 in term 1.
I20260807 10:49:44.367993 5821 consensus_queue.cc:237] T eea67a9585694b11a0fb56c3bc4510ca P 59c8fcaf1fd44e3d8fb950f658162cb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } }
I20260807 10:49:44.370725 5868 tablet_replica.cc:439] T eea67a9585694b11a0fb56c3bc4510ca P 59c8fcaf1fd44e3d8fb950f658162cb0: TxnStatusTablet state changed. Reason: New leader 59c8fcaf1fd44e3d8fb950f658162cb0. Latest consensus state: current_term: 1 leader_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } } }
I20260807 10:49:44.371279 5868 tablet_replica.cc:442] T eea67a9585694b11a0fb56c3bc4510ca P 59c8fcaf1fd44e3d8fb950f658162cb0: This TxnStatusTablet replica's current role is: LEADER
I20260807 10:49:44.372318 5887 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 10:49:44.372675 5887 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 10:49:44.373493 5067 catalog_manager.cc:5495] T eea67a9585694b11a0fb56c3bc4510ca P 59c8fcaf1fd44e3d8fb950f658162cb0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 59c8fcaf1fd44e3d8fb950f658162cb0 (127.3.73.194). New cstate: current_term: 1 leader_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } health_report { overall_health: UNKNOWN } } }
I20260807 10:49:44.423152 5879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:44.399730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:34367 (local address 127.0.0.1:49104)
0807 10:49:44.400233 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:44.400256 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:44.400412 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:44.413686 (+ 13274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:44.413693 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:44.413711 (+ 18us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 10:49:44.414177 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:44.414196 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:44.417296 (+ 3100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:44.417302 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:44.419441 (+ 2139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:44.419463 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:44.420891 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:44.420900 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:44.421144 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:44.421223 (+ 79us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 10:49:44.422370 (+ 1147us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 10:49:44.422401 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 10:49:44.422504 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 10:49:44.422716 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321}
I20260807 10:49:44.423326 5893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:44.399900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49104 (local address 127.3.73.194:34367)
0807 10:49:44.413099 (+ 13199us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:44.413106 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:44.413137 (+ 31us) server_negotiation.cc:407] Connection header received
0807 10:49:44.413252 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:44.413258 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:44.413368 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:44.414371 (+ 1003us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:49:44.414982 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:44.417022 (+ 2040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:44.419687 (+ 2665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:44.420608 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:44.420817 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:44.421527 (+ 710us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:49:44.422236 (+ 709us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:49:44.422408 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:44.422527 (+ 119us) server_negotiation.cc:299] Negotiation successful
0807 10:49:44.422624 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13006,"thread_start_us":12723,"threads_started":1}
I20260807 10:49:44.431830 5714 raft_consensus.cc:1261] T eea67a9585694b11a0fb56c3bc4510ca P 1c06edc993584b45a6fd20b3d6cb4ded [term 1 FOLLOWER]: Refusing update from remote peer 59c8fcaf1fd44e3d8fb950f658162cb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:49:44.432435 5304 raft_consensus.cc:1261] T eea67a9585694b11a0fb56c3bc4510ca P ec508c2f08974d0b8c83a369c0d40616 [term 1 FOLLOWER]: Refusing update from remote peer 59c8fcaf1fd44e3d8fb950f658162cb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:49:44.432870 5821 consensus_queue.cc:1035] T eea67a9585694b11a0fb56c3bc4510ca P 59c8fcaf1fd44e3d8fb950f658162cb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:49:44.433583 5868 consensus_queue.cc:1035] T eea67a9585694b11a0fb56c3bc4510ca P 59c8fcaf1fd44e3d8fb950f658162cb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:49:44.455627 5824 tablet_replica.cc:439] T eea67a9585694b11a0fb56c3bc4510ca P 1c06edc993584b45a6fd20b3d6cb4ded: TxnStatusTablet state changed. Reason: New leader 59c8fcaf1fd44e3d8fb950f658162cb0. Latest consensus state: current_term: 1 leader_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } } }
I20260807 10:49:44.456205 5824 tablet_replica.cc:442] T eea67a9585694b11a0fb56c3bc4510ca P 1c06edc993584b45a6fd20b3d6cb4ded: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 10:49:44.458791 5895 mvcc.cc:205] Tried to move back new op lower bound from 7315864717021396992 to 7315864716774502400. Current Snapshot: MvccSnapshot[applied={T|T < 7315864717021396992}]
I20260807 10:49:44.465559 5821 tablet_replica.cc:439] T eea67a9585694b11a0fb56c3bc4510ca P 59c8fcaf1fd44e3d8fb950f658162cb0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } } }
I20260807 10:49:44.466380 5821 tablet_replica.cc:442] T eea67a9585694b11a0fb56c3bc4510ca P 59c8fcaf1fd44e3d8fb950f658162cb0: This TxnStatusTablet replica's current role is: LEADER
I20260807 10:49:44.488286 5837 tablet_replica.cc:439] T eea67a9585694b11a0fb56c3bc4510ca P ec508c2f08974d0b8c83a369c0d40616: TxnStatusTablet state changed. Reason: New leader 59c8fcaf1fd44e3d8fb950f658162cb0. Latest consensus state: current_term: 1 leader_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } } }
I20260807 10:49:44.488970 5837 tablet_replica.cc:442] T eea67a9585694b11a0fb56c3bc4510ca P ec508c2f08974d0b8c83a369c0d40616: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 10:49:44.491820 5868 tablet_replica.cc:439] T eea67a9585694b11a0fb56c3bc4510ca P 59c8fcaf1fd44e3d8fb950f658162cb0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } } }
I20260807 10:49:44.492249 5868 tablet_replica.cc:442] T eea67a9585694b11a0fb56c3bc4510ca P 59c8fcaf1fd44e3d8fb950f658162cb0: This TxnStatusTablet replica's current role is: LEADER
I20260807 10:49:44.494998 5899 mvcc.cc:205] Tried to move back new op lower bound from 7315864717021396992 to 7315864716774502400. Current Snapshot: MvccSnapshot[applied={T|T < 7315864717021396992}]
I20260807 10:49:44.507584 5893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:44.479062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49112 (local address 127.3.73.194:34367)
0807 10:49:44.479282 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:44.479289 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:44.497775 (+ 18486us) server_negotiation.cc:407] Connection header received
0807 10:49:44.498101 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:44.498108 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:44.498234 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:44.498307 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:49:44.499490 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:44.501176 (+ 1686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:44.504668 (+ 3492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:44.505627 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:44.505742 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:44.506513 (+ 771us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:49:44.506679 (+ 166us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:49:44.506758 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:44.507044 (+ 286us) server_negotiation.cc:299] Negotiation successful
0807 10:49:44.507155 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 10:49:44.508580 5824 tablet_replica.cc:439] T eea67a9585694b11a0fb56c3bc4510ca P 1c06edc993584b45a6fd20b3d6cb4ded: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } } }
I20260807 10:49:44.509038 5824 tablet_replica.cc:442] T eea67a9585694b11a0fb56c3bc4510ca P 1c06edc993584b45a6fd20b3d6cb4ded: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 10:49:44.520987 5837 tablet_replica.cc:439] T eea67a9585694b11a0fb56c3bc4510ca P ec508c2f08974d0b8c83a369c0d40616: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 } } peers { permanent_uuid: "59c8fcaf1fd44e3d8fb950f658162cb0" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34367 } } peers { permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 } } }
I20260807 10:49:44.521287 5837 tablet_replica.cc:442] T eea67a9585694b11a0fb56c3bc4510ca P ec508c2f08974d0b8c83a369c0d40616: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 10:49:44.554936 5893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:44.541549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:36421 (local address 127.3.73.194:34367)
0807 10:49:44.541789 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:44.541796 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:44.543160 (+ 1364us) server_negotiation.cc:407] Connection header received
0807 10:49:44.543408 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:44.543414 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:44.543536 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:44.543643 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:49:44.544812 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:44.546649 (+ 1837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:44.551052 (+ 4403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:44.552122 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:44.552350 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:44.553398 (+ 1048us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:49:44.553549 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:49:44.553649 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:44.554018 (+ 369us) server_negotiation.cc:299] Negotiation successful
0807 10:49:44.554111 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 10:49:44.555576 5911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:44.541779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:34367 (local address 127.3.73.194:36421)
0807 10:49:44.543009 (+ 1230us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:44.543035 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:44.543241 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:44.543767 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:44.543773 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:44.543786 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 10:49:44.544564 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:44.544579 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:44.546864 (+ 2285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:44.546869 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:44.550832 (+ 3963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:44.550850 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:44.552584 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:44.552595 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:44.553011 (+ 416us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:44.553134 (+ 123us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 10:49:44.553682 (+ 548us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 10:49:44.553714 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 10:49:44.553995 (+ 281us) client_negotiation.cc:240] Negotiation successful
0807 10:49:44.554209 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1000,"thread_start_us":806,"threads_started":1}
I20260807 10:49:44.575564 5714 raft_consensus.cc:1261] T bab91472e2694999900fed238ee19ead P 1c06edc993584b45a6fd20b3d6cb4ded [term 1 FOLLOWER]: Refusing update from remote peer 59c8fcaf1fd44e3d8fb950f658162cb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:49:44.576470 5868 consensus_queue.cc:1035] T bab91472e2694999900fed238ee19ead P 59c8fcaf1fd44e3d8fb950f658162cb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c06edc993584b45a6fd20b3d6cb4ded" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 34257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:49:44.580467 5303 raft_consensus.cc:1261] T bab91472e2694999900fed238ee19ead P ec508c2f08974d0b8c83a369c0d40616 [term 1 FOLLOWER]: Refusing update from remote peer 59c8fcaf1fd44e3d8fb950f658162cb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:49:44.581542 5868 consensus_queue.cc:1035] T bab91472e2694999900fed238ee19ead P 59c8fcaf1fd44e3d8fb950f658162cb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec508c2f08974d0b8c83a369c0d40616" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:49:44.687594 5067 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:49:44.688109 5067 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:49:44.692004 5067 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9a4a5d57a3a44ef6b4d184ffd17c4fd7: Sending DeleteTablet for 3 replicas of tablet bab91472e2694999900fed238ee19ead
I20260807 10:49:44.693374 5682 tablet_service.cc:1514] Processing DeleteTablet for tablet bab91472e2694999900fed238ee19ead with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:49:44 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:46548
I20260807 10:49:44.694655 5263 tablet_service.cc:1514] Processing DeleteTablet for tablet bab91472e2694999900fed238ee19ead with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:49:44 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:50326
I20260807 10:49:44.700769 5476 tablet_service.cc:1514] Processing DeleteTablet for tablet bab91472e2694999900fed238ee19ead with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:49:44 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:49090
I20260807 10:49:44.702347 5922 tablet_replica.cc:330] T bab91472e2694999900fed238ee19ead P 59c8fcaf1fd44e3d8fb950f658162cb0: stopping tablet replica
I20260807 10:49:44.702740 5922 raft_consensus.cc:2229] T bab91472e2694999900fed238ee19ead P 59c8fcaf1fd44e3d8fb950f658162cb0 [term 1 LEADER]: Raft consensus shutting down.
I20260807 10:49:44.703679 5922 raft_consensus.cc:2258] T bab91472e2694999900fed238ee19ead P 59c8fcaf1fd44e3d8fb950f658162cb0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:49:44.707244 5922 ts_tablet_manager.cc:1905] T bab91472e2694999900fed238ee19ead P 59c8fcaf1fd44e3d8fb950f658162cb0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:49:44.711556 5919 tablet_replica.cc:330] T bab91472e2694999900fed238ee19ead P 1c06edc993584b45a6fd20b3d6cb4ded: stopping tablet replica
I20260807 10:49:44.711977 5919 raft_consensus.cc:2229] T bab91472e2694999900fed238ee19ead P 1c06edc993584b45a6fd20b3d6cb4ded [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:49:44.712319 5919 raft_consensus.cc:2258] T bab91472e2694999900fed238ee19ead P 1c06edc993584b45a6fd20b3d6cb4ded [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:49:44.713902 5919 ts_tablet_manager.cc:1905] T bab91472e2694999900fed238ee19ead P 1c06edc993584b45a6fd20b3d6cb4ded: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:49:44.723433 5920 tablet_replica.cc:330] T bab91472e2694999900fed238ee19ead P ec508c2f08974d0b8c83a369c0d40616: stopping tablet replica
I20260807 10:49:44.723780 5920 raft_consensus.cc:2229] T bab91472e2694999900fed238ee19ead P ec508c2f08974d0b8c83a369c0d40616 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:49:44.724350 5920 raft_consensus.cc:2258] T bab91472e2694999900fed238ee19ead P ec508c2f08974d0b8c83a369c0d40616 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:49:44.725965 5920 ts_tablet_manager.cc:1905] T bab91472e2694999900fed238ee19ead P ec508c2f08974d0b8c83a369c0d40616: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:49:44.726954 5919 ts_tablet_manager.cc:1918] T bab91472e2694999900fed238ee19ead P 1c06edc993584b45a6fd20b3d6cb4ded: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 10:49:44.727248 5919 log.cc:1196] T bab91472e2694999900fed238ee19ead P 1c06edc993584b45a6fd20b3d6cb4ded: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-2/wal/wals/bab91472e2694999900fed238ee19ead
I20260807 10:49:44.732945 5919 ts_tablet_manager.cc:1939] T bab91472e2694999900fed238ee19ead P 1c06edc993584b45a6fd20b3d6cb4ded: Deleting consensus metadata
I20260807 10:49:44.734926 5045 catalog_manager.cc:4841] TS 1c06edc993584b45a6fd20b3d6cb4ded (127.3.73.195:34257): tablet bab91472e2694999900fed238ee19ead (table test-table [id=1c81e0f9210f45e2b37a5a042dd1895c]) successfully deleted
I20260807 10:49:44.738864 5920 ts_tablet_manager.cc:1918] T bab91472e2694999900fed238ee19ead P ec508c2f08974d0b8c83a369c0d40616: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 10:49:44.739446 5920 log.cc:1196] T bab91472e2694999900fed238ee19ead P ec508c2f08974d0b8c83a369c0d40616: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-0/wal/wals/bab91472e2694999900fed238ee19ead
I20260807 10:49:44.740491 5920 ts_tablet_manager.cc:1939] T bab91472e2694999900fed238ee19ead P ec508c2f08974d0b8c83a369c0d40616: Deleting consensus metadata
I20260807 10:49:44.742471 5046 catalog_manager.cc:4841] TS ec508c2f08974d0b8c83a369c0d40616 (127.3.73.193:44329): tablet bab91472e2694999900fed238ee19ead (table test-table [id=1c81e0f9210f45e2b37a5a042dd1895c]) successfully deleted
I20260807 10:49:44.744376 5922 ts_tablet_manager.cc:1918] T bab91472e2694999900fed238ee19ead P 59c8fcaf1fd44e3d8fb950f658162cb0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 10:49:44.744659 5922 log.cc:1196] T bab91472e2694999900fed238ee19ead P 59c8fcaf1fd44e3d8fb950f658162cb0: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786099767749326-3367-0/minicluster-data/ts-1/wal/wals/bab91472e2694999900fed238ee19ead
I20260807 10:49:44.746259 5922 ts_tablet_manager.cc:1939] T bab91472e2694999900fed238ee19ead P 59c8fcaf1fd44e3d8fb950f658162cb0: Deleting consensus metadata
I20260807 10:49:44.747643 5045 catalog_manager.cc:4841] TS 59c8fcaf1fd44e3d8fb950f658162cb0 (127.3.73.194:34367): tablet bab91472e2694999900fed238ee19ead (table test-table [id=1c81e0f9210f45e2b37a5a042dd1895c]) successfully deleted
Aug 07 10:49:44 dist-test-slave-x5fp krb5kdc[4996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099783, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.73.193@KRBTEST.COM
I20260807 10:49:44.785243 5923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:44.728302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50352 (local address 127.3.73.193:44329)
0807 10:49:44.749007 (+ 20705us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:44.749018 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:44.750570 (+ 1552us) server_negotiation.cc:407] Connection header received
0807 10:49:44.750912 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:44.750920 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:44.751124 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:44.751368 (+ 244us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:44.756332 (+ 4964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:44.758891 (+ 2559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:44.761610 (+ 2719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:44.762787 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:44.762942 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:44.771255 (+ 8313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:44.771273 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:44.771295 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:44.771394 (+ 99us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:44.780057 (+ 8663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:44.781476 (+ 1419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:44.781488 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:44.781492 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:44.781860 (+ 368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:44.782479 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:44.782485 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:44.782489 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:44.783207 (+ 718us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:44.783769 (+ 562us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:44.784024 (+ 255us) server_negotiation.cc:299] Negotiation successful
0807 10:49:44.784511 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20486,"thread_start_us":20722,"threads_started":1}
W20260807 10:49:44.786463 5364 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50352
I20260807 10:49:44.835855 5927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:44.793763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53128 (local address 127.3.73.254:41707)
0807 10:49:44.806230 (+ 12467us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:44.806237 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:44.809636 (+ 3399us) server_negotiation.cc:407] Connection header received
0807 10:49:44.810006 (+ 370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:44.810012 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:44.810163 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:44.810342 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:44.813393 (+ 3051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:44.815187 (+ 1794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:44.817825 (+ 2638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:44.819197 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:44.819432 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:44.824804 (+ 5372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:44.824813 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:44.824819 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:44.824868 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:44.831131 (+ 6263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:44.832388 (+ 1257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:44.832396 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:44.832400 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:44.832714 (+ 314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:44.833460 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:44.833466 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:44.833470 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:44.834039 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:44.834479 (+ 440us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:44.834864 (+ 385us) server_negotiation.cc:299] Negotiation successful
0807 10:49:44.835189 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12308,"thread_start_us":12479,"threads_started":1}
W20260807 10:49:44.837108 5067 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53128
I20260807 10:49:44.840562 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 5164
I20260807 10:49:44.862133 5923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:44.841526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50364 (local address 127.3.73.193:44329)
0807 10:49:44.843255 (+ 1729us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:44.843261 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:44.843281 (+ 20us) server_negotiation.cc:407] Connection header received
0807 10:49:44.843350 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:44.843355 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:44.843455 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:44.843570 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:44.844970 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:44.846233 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:44.847994 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:44.848779 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:44.848928 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:44.852599 (+ 3671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:44.852606 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:44.852611 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:44.852663 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:44.857797 (+ 5134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:44.858830 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:44.858836 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:44.858840 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:44.859154 (+ 314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:44.859724 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:44.859745 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:44.859749 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:44.860221 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:44.860678 (+ 457us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:44.860962 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 10:49:44.861461 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1584}
W20260807 10:49:44.862872 5364 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50364
W20260807 10:49:44.863770 3367 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 5164: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 10:49:44.864156 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 5164
I20260807 10:49:44.899664 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 5376
Aug 07 10:49:44 dist-test-slave-x5fp krb5kdc[4996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099783, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.73.194@KRBTEST.COM
I20260807 10:49:44.923588 5893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:44.900592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49126 (local address 127.3.73.194:34367)
0807 10:49:44.900851 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:44.900857 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:44.900882 (+ 25us) server_negotiation.cc:407] Connection header received
0807 10:49:44.901070 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:44.901075 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:44.901145 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:44.901246 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:44.903313 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:44.904129 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:44.906655 (+ 2526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:44.907228 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:44.907373 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:44.915959 (+ 8586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:44.915978 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:44.915995 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:44.916056 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:44.919761 (+ 3705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:44.921068 (+ 1307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:44.921077 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:44.921080 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:44.921266 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:44.921864 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:44.921867 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:44.921870 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:44.922316 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:44.922611 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:44.923101 (+ 490us) server_negotiation.cc:299] Negotiation successful
0807 10:49:44.923309 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260807 10:49:44.924850 5574 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49126
W20260807 10:49:44.925976 3367 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 5376: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 10:49:44.926404 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 5376
I20260807 10:49:44.959141 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 5585
Aug 07 10:49:44 dist-test-slave-x5fp krb5kdc[4996](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099783, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.73.195@KRBTEST.COM
I20260807 10:49:44.981278 5929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:44.960149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46554 (local address 127.3.73.195:34257)
0807 10:49:44.960705 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:44.960710 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:44.960769 (+ 59us) server_negotiation.cc:407] Connection header received
0807 10:49:44.960835 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:44.960839 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:44.960904 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:44.961020 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:44.962368 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:44.963158 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:44.964598 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:44.965442 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:44.965587 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:44.972029 (+ 6442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:44.972045 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:44.972063 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:44.972137 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:44.978066 (+ 5929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:44.978923 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:44.978934 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:44.978938 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:44.979098 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:44.979519 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:44.979524 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:44.979528 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:44.979940 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:44.980265 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:44.980599 (+ 334us) server_negotiation.cc:299] Negotiation successful
0807 10:49:44.980913 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":308,"threads_started":1}
W20260807 10:49:44.982043 5776 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46554
W20260807 10:49:44.982923 3367 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 5585: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 10:49:44.983121 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 5585
I20260807 10:49:45.016268 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 5014
I20260807 10:49:45.017858 5120 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:49:45.145258 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 5014
2026-08-07T10:49:45Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5788 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:49:45 dist-test-slave-x5fp krb5kdc[5938](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:49:45 dist-test-slave-x5fp krb5kdc[5938](info): set up 2 sockets
Aug 07 10:49:45 dist-test-slave-x5fp krb5kdc[5938](info): commencing operation
krb5kdc: starting...
W20260807 10:49:46.484094 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.282s user 0.003s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:49:46 dist-test-slave-x5fp krb5kdc[5938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:49:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:49:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.73.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
I20260807 10:49:46.738669 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:41773
--webserver_interface=127.3.73.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:36429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:41773
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:49:47.090776 5959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:49:47.091136 5959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:49:47.091246 5959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:49:47.099740 5959 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:49:47.099910 5959 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 10:49:47.099962 5959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:49:47.100030 5959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:49:47.100092 5959 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:49:47.100162 5959 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:49:47.111560 5959 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:36429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:41773
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:41773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:49:47.113091 5959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:49:47.114509 5959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:49:47.127924 5967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:47.127887 5971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:47.127978 5968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:49:47.128883 5959 server_base.cc:1034] running on GCE node
I20260807 10:49:47.129709 5959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:49:47.131224 5959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:49:47.132513 5959 hybrid_clock.cc:648] HybridClock initialized: now 1786099787132439 us; error 55 us; skew 500 ppm
I20260807 10:49:47.132969 5959 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:49:47 dist-test-slave-x5fp krb5kdc[5938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099787, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:49:47.141706 5959 init.cc:377] Logged in from keytab as kudu/127.3.73.254@KRBTEST.COM (short username kudu)
I20260807 10:49:47.143708 5959 webserver.cc:460] Webserver started at http://127.3.73.254:38179/ using document root <none> and password file <none>
I20260807 10:49:47.144547 5959 fs_manager.cc:359] Metadata directory not provided
I20260807 10:49:47.144699 5959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:49:47.145128 5959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:49:47.148510 5959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "8ea443e2ab0a47f1b65df7aeac9255e8"
format_stamp: "Formatted at 2026-08-07 10:49:47 on dist-test-slave-x5fp"
server_key: "d05e02f6c79ffea48face614745a70f0"
server_key_iv: "f02b0f10ca71a6117687dbd99240f4f7"
server_key_version: "encryptionkey@0"
I20260807 10:49:47.149434 5959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "8ea443e2ab0a47f1b65df7aeac9255e8"
format_stamp: "Formatted at 2026-08-07 10:49:47 on dist-test-slave-x5fp"
server_key: "d05e02f6c79ffea48face614745a70f0"
server_key_iv: "f02b0f10ca71a6117687dbd99240f4f7"
server_key_version: "encryptionkey@0"
I20260807 10:49:47.155296 5959 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260807 10:49:47.159386 5983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:47.161106 5959 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 10:49:47.161335 5959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "8ea443e2ab0a47f1b65df7aeac9255e8"
format_stamp: "Formatted at 2026-08-07 10:49:47 on dist-test-slave-x5fp"
server_key: "d05e02f6c79ffea48face614745a70f0"
server_key_iv: "f02b0f10ca71a6117687dbd99240f4f7"
server_key_version: "encryptionkey@0"
I20260807 10:49:47.161595 5959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:49:47.178705 5959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:49:47.184180 5959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:49:47.184556 5959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:49:47.209285 5959 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:41773
I20260807 10:49:47.209332 6078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:41773 every 8 connection(s)
I20260807 10:49:47.210762 5959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:49:47.215139 6081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:49:47.215572 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 5959
I20260807 10:49:47.215842 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:49:47.216830 3367 external_mini_cluster.cc:1428] Setting key fa7428dcedb5d48ea586cc3e5e705ada
I20260807 10:49:47.231979 6081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ea443e2ab0a47f1b65df7aeac9255e8: Bootstrap starting.
I20260807 10:49:47.236529 6081 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ea443e2ab0a47f1b65df7aeac9255e8: Neither blocks nor log segments found. Creating new log.
Aug 07 10:49:47 dist-test-slave-x5fp krb5kdc[5938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:49:47.237913 6081 log.cc:824] T 00000000000000000000000000000000 P 8ea443e2ab0a47f1b65df7aeac9255e8: Log is configured to *not* fsync() on all Append() calls
I20260807 10:49:47.242735 6081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ea443e2ab0a47f1b65df7aeac9255e8: No bootstrap required, opened a new log
I20260807 10:49:47.250152 6085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:47.219291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46872 (local address 127.3.73.254:41773)
0807 10:49:47.222843 (+ 3552us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:47.222863 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:47.223402 (+ 539us) server_negotiation.cc:407] Connection header received
0807 10:49:47.225523 (+ 2121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:47.225549 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:47.225983 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:47.226346 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:47.229001 (+ 2655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:47.230496 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:47.232324 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:47.233138 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:47.233323 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:47.238496 (+ 5173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:47.238512 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:47.238541 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:47.238625 (+ 84us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:47.244199 (+ 5574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:47.245512 (+ 1313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:47.245521 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:47.245531 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:47.245782 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:47.246227 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:47.246232 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:47.246236 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:47.247141 (+ 905us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:47.247481 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:47.247979 (+ 498us) server_negotiation.cc:299] Negotiation successful
0807 10:49:47.248506 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3183,"thread_start_us":2853,"threads_started":1}
I20260807 10:49:47.254765 6081 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ea443e2ab0a47f1b65df7aeac9255e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ea443e2ab0a47f1b65df7aeac9255e8" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41773 } }
I20260807 10:49:47.255146 6081 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ea443e2ab0a47f1b65df7aeac9255e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:49:47.255223 6081 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ea443e2ab0a47f1b65df7aeac9255e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ea443e2ab0a47f1b65df7aeac9255e8, State: Initialized, Role: FOLLOWER
I20260807 10:49:47.255797 6081 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ea443e2ab0a47f1b65df7aeac9255e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ea443e2ab0a47f1b65df7aeac9255e8" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41773 } }
I20260807 10:49:47.256013 6081 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ea443e2ab0a47f1b65df7aeac9255e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:49:47.256137 6081 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ea443e2ab0a47f1b65df7aeac9255e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:49:47.256290 6081 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ea443e2ab0a47f1b65df7aeac9255e8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:49:47.258991 6081 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ea443e2ab0a47f1b65df7aeac9255e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ea443e2ab0a47f1b65df7aeac9255e8" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41773 } }
I20260807 10:49:47.259451 6081 leader_election.cc:302] T 00000000000000000000000000000000 P 8ea443e2ab0a47f1b65df7aeac9255e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ea443e2ab0a47f1b65df7aeac9255e8; no voters:
I20260807 10:49:47.260068 6081 leader_election.cc:288] T 00000000000000000000000000000000 P 8ea443e2ab0a47f1b65df7aeac9255e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:49:47.260262 6089 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ea443e2ab0a47f1b65df7aeac9255e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:49:47.261471 6089 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ea443e2ab0a47f1b65df7aeac9255e8 [term 1 LEADER]: Becoming Leader. State: Replica: 8ea443e2ab0a47f1b65df7aeac9255e8, State: Running, Role: LEADER
I20260807 10:49:47.262143 6089 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ea443e2ab0a47f1b65df7aeac9255e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ea443e2ab0a47f1b65df7aeac9255e8" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41773 } }
I20260807 10:49:47.262586 6081 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ea443e2ab0a47f1b65df7aeac9255e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:49:47.265260 6091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ea443e2ab0a47f1b65df7aeac9255e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ea443e2ab0a47f1b65df7aeac9255e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ea443e2ab0a47f1b65df7aeac9255e8" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41773 } } }
I20260807 10:49:47.265203 6092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ea443e2ab0a47f1b65df7aeac9255e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ea443e2ab0a47f1b65df7aeac9255e8. Latest consensus state: current_term: 1 leader_uuid: "8ea443e2ab0a47f1b65df7aeac9255e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ea443e2ab0a47f1b65df7aeac9255e8" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41773 } } }
I20260807 10:49:47.265776 6091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ea443e2ab0a47f1b65df7aeac9255e8 [sys.catalog]: This master's current role is: LEADER
I20260807 10:49:47.265831 6092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ea443e2ab0a47f1b65df7aeac9255e8 [sys.catalog]: This master's current role is: LEADER
I20260807 10:49:47.271073 6098 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:49:47.276928 6098 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:49:47.287554 6098 catalog_manager.cc:1347] Generated new cluster ID: 7a6cbd23c2874f92a80df3615716849f
I20260807 10:49:47.287688 6098 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:49:47.314777 6098 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:49:47.316583 6098 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:49:47.324512 6098 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ea443e2ab0a47f1b65df7aeac9255e8: Generated new TSK 0
I20260807 10:49:47.325721 6098 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.73.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab.
Entry for principal kudu/127.3.73.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab.
I20260807 10:49:47.398272 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.193:0
--local_ip_for_outbound_sockets=127.3.73.193
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:41773
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:36429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:49:47.760696 6118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:49:47.761053 6118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:49:47.761216 6118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:49:47.768814 6118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:49:47.768949 6118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:49:47.769105 6118 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 10:49:47.769165 6118 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 10:49:47.769289 6118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.193
I20260807 10:49:47.779470 6118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:36429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.73.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.73.254:41773
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.193
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:49:47.780864 6118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:49:47.782226 6118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:49:47.796155 6129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:47.796193 6138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:47.797044 6140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:49:47.797510 6118 server_base.cc:1034] running on GCE node
I20260807 10:49:47.798207 6118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:49:47.799999 6118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:49:47.801294 6118 hybrid_clock.cc:648] HybridClock initialized: now 1786099787801229 us; error 49 us; skew 500 ppm
I20260807 10:49:47.801733 6118 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:49:47 dist-test-slave-x5fp krb5kdc[5938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099787, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:49:47.805292 6118 init.cc:377] Logged in from keytab as kudu/127.3.73.193@KRBTEST.COM (short username kudu)
I20260807 10:49:47.807183 6118 webserver.cc:460] Webserver started at http://127.3.73.193:34195/ using document root <none> and password file <none>
I20260807 10:49:47.808045 6118 fs_manager.cc:359] Metadata directory not provided
I20260807 10:49:47.808226 6118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:49:47.808607 6118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:49:47.811733 6118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-0/data/instance:
uuid: "ea7e70332cf64545889d381b59a7d558"
format_stamp: "Formatted at 2026-08-07 10:49:47 on dist-test-slave-x5fp"
server_key: "48a1ef6934f9a97e14740ec1befa4e00"
server_key_iv: "d32e38b8279ca022f333e5ce379e1a31"
server_key_version: "encryptionkey@0"
I20260807 10:49:47.812500 6118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance:
uuid: "ea7e70332cf64545889d381b59a7d558"
format_stamp: "Formatted at 2026-08-07 10:49:47 on dist-test-slave-x5fp"
server_key: "48a1ef6934f9a97e14740ec1befa4e00"
server_key_iv: "d32e38b8279ca022f333e5ce379e1a31"
server_key_version: "encryptionkey@0"
I20260807 10:49:47.818362 6118 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 10:49:47.822523 6150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:47.824234 6118 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 10:49:47.824395 6118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-0/wal
uuid: "ea7e70332cf64545889d381b59a7d558"
format_stamp: "Formatted at 2026-08-07 10:49:47 on dist-test-slave-x5fp"
server_key: "48a1ef6934f9a97e14740ec1befa4e00"
server_key_iv: "d32e38b8279ca022f333e5ce379e1a31"
server_key_version: "encryptionkey@0"
I20260807 10:49:47.824642 6118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:49:47.842214 6118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:49:47.847401 6118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:49:47.847754 6118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:49:47.851047 6118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:49:47.851195 6118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:47.851394 6118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:49:47.851492 6118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 10:49:47 dist-test-slave-x5fp krb5kdc[5938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099787, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:49:47.878997 6162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:47.851617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:56899 (local address 127.3.73.254:41773)
0807 10:49:47.854229 (+ 2612us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:47.854236 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:47.854262 (+ 26us) server_negotiation.cc:407] Connection header received
0807 10:49:47.856265 (+ 2003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:47.856272 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:47.856384 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:47.856521 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:47.859125 (+ 2604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:47.860486 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:47.863472 (+ 2986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:47.864255 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:47.864467 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:47.870186 (+ 5719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:47.870197 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:47.870203 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:47.870265 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:47.874666 (+ 4401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:47.875719 (+ 1053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:47.875729 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:47.875735 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:47.875935 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:47.876474 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:47.876483 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:47.876489 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:47.877001 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:47.877172 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:47.878083 (+ 911us) server_negotiation.cc:299] Negotiation successful
0807 10:49:47.878442 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2453,"thread_start_us":2331,"threads_started":1}
I20260807 10:49:47.879679 6163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:47.852193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41773 (local address 127.3.73.193:56899)
0807 10:49:47.853460 (+ 1267us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:47.853547 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:47.855930 (+ 2383us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:47.856852 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:47.856869 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:47.857818 (+ 949us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:49:47.858838 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:47.858878 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:47.860697 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:47.860701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:47.863305 (+ 2604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:47.863320 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:47.864594 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:47.864601 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:47.864928 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:47.866117 (+ 1189us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:49:47.866174 (+ 57us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:49:47.869761 (+ 3587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:49:47.874895 (+ 5134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:47.874909 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:47.874937 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:47.875539 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:47.876088 (+ 549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:47.876092 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:47.876095 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:47.876338 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:47.877162 (+ 824us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:49:47.877172 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:49:47.877622 (+ 450us) client_negotiation.cc:769] Sending connection context
0807 10:49:47.877881 (+ 259us) client_negotiation.cc:240] Negotiation successful
0807 10:49:47.878343 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":835,"thread_start_us":511,"threads_started":1}
I20260807 10:49:47.902516 6162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:47.887611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:48031 (local address 127.3.73.254:41773)
0807 10:49:47.887890 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:47.887899 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:47.888568 (+ 669us) server_negotiation.cc:407] Connection header received
0807 10:49:47.888807 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:47.888817 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:47.888945 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:47.889072 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:47.890404 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:47.891137 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:47.892524 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:47.893020 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:47.893166 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:47.895130 (+ 1964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:47.895135 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:47.895139 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:47.895172 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:47.898978 (+ 3806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:47.900107 (+ 1129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:47.900116 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:47.900122 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:47.900336 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:47.900845 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:47.900852 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:47.900855 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:47.901223 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:47.901401 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:47.901886 (+ 485us) server_negotiation.cc:299] Negotiation successful
0807 10:49:47.902145 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 10:49:47.902727 6265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:47.887558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41773 (local address 127.3.73.193:48031)
0807 10:49:47.888435 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:47.888457 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:47.888612 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:47.889119 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:47.889122 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:47.889734 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:49:47.890223 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:47.890235 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:47.891308 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:47.891313 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:47.892359 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:47.892370 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:47.893229 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:47.893235 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:47.893415 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:47.894108 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:49:47.894127 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:49:47.894912 (+ 785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:49:47.899234 (+ 4322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:47.899245 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:47.899251 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:47.899936 (+ 685us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:47.900495 (+ 559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:47.900500 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:47.900504 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:47.900657 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:47.901381 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:49:47.901385 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:49:47.901599 (+ 214us) client_negotiation.cc:769] Sending connection context
0807 10:49:47.901753 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 10:49:47.902225 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"thread_start_us":513,"threads_started":1}
I20260807 10:49:47.914433 6118 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.193:44797
I20260807 10:49:47.914466 6328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.193:44797 every 8 connection(s)
I20260807 10:49:47.915792 6118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
I20260807 10:49:47.916317 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 6118
I20260807 10:49:47.916582 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance
I20260807 10:49:47.917634 3367 external_mini_cluster.cc:1428] Setting key 628bc5431ed383543e5e24eb94d0642a
I20260807 10:49:47.918761 6329 heartbeater.cc:347] Connected to a master server at 127.3.73.254:41773
I20260807 10:49:47.919019 6329 heartbeater.cc:464] Registering TS with master...
I20260807 10:49:47.919929 6329 heartbeater.cc:511] Master 127.3.73.254:41773 requested a full tablet report, sending...
I20260807 10:49:47.922829 6013 ts_manager.cc:194] Registered new tserver with Master: ea7e70332cf64545889d381b59a7d558 (127.3.73.193:44797)
I20260807 10:49:47.925331 6013 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.193@KRBTEST.COM'} at 127.3.73.193:56899
WARNING: no policy specified for kudu/127.3.73.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab.
Entry for principal kudu/127.3.73.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab.
I20260807 10:49:47.988090 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.194:0
--local_ip_for_outbound_sockets=127.3.73.194
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:41773
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:36429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:49:48.347442 6334 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:49:48.347862 6334 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:49:48.348053 6334 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:49:48.355845 6334 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:49:48.356015 6334 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:49:48.356178 6334 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 10:49:48.356271 6334 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 10:49:48.356395 6334 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.194
I20260807 10:49:48.367033 6334 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:36429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.73.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.73.254:41773
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.194
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:49:48.368458 6334 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:49:48.369908 6334 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:49:48.384558 6341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:48.384577 6344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:48.384657 6340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:49:48.385721 6334 server_base.cc:1034] running on GCE node
I20260807 10:49:48.386535 6334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:49:48.387696 6334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:49:48.388990 6334 hybrid_clock.cc:648] HybridClock initialized: now 1786099788388930 us; error 71 us; skew 500 ppm
I20260807 10:49:48.389384 6334 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:49:48 dist-test-slave-x5fp krb5kdc[5938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099788, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:49:48.393242 6334 init.cc:377] Logged in from keytab as kudu/127.3.73.194@KRBTEST.COM (short username kudu)
I20260807 10:49:48.395074 6334 webserver.cc:460] Webserver started at http://127.3.73.194:33777/ using document root <none> and password file <none>
I20260807 10:49:48.395859 6334 fs_manager.cc:359] Metadata directory not provided
I20260807 10:49:48.396042 6334 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:49:48.396409 6334 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:49:48.399955 6334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-1/data/instance:
uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a"
format_stamp: "Formatted at 2026-08-07 10:49:48 on dist-test-slave-x5fp"
server_key: "bddf5d9bb3df2e248940659bb3048004"
server_key_iv: "005eb7d24e464ba64af0c7af1007680c"
server_key_version: "encryptionkey@0"
I20260807 10:49:48.400962 6334 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance:
uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a"
format_stamp: "Formatted at 2026-08-07 10:49:48 on dist-test-slave-x5fp"
server_key: "bddf5d9bb3df2e248940659bb3048004"
server_key_iv: "005eb7d24e464ba64af0c7af1007680c"
server_key_version: "encryptionkey@0"
I20260807 10:49:48.407245 6334 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 10:49:48.411414 6351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:48.413179 6334 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 10:49:48.413385 6334 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-1/wal
uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a"
format_stamp: "Formatted at 2026-08-07 10:49:48 on dist-test-slave-x5fp"
server_key: "bddf5d9bb3df2e248940659bb3048004"
server_key_iv: "005eb7d24e464ba64af0c7af1007680c"
server_key_version: "encryptionkey@0"
I20260807 10:49:48.413625 6334 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:49:48.443472 6334 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:49:48.448923 6334 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:49:48.449280 6334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:49:48.452502 6334 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:49:48.452656 6334 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:48.452826 6334 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:49:48.452976 6334 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 10:49:48 dist-test-slave-x5fp krb5kdc[5938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099788, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:49:48.481921 6367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:48.453324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:42503 (local address 127.3.73.254:41773)
0807 10:49:48.454034 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:48.454040 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:48.455210 (+ 1170us) server_negotiation.cc:407] Connection header received
0807 10:49:48.457678 (+ 2468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:48.457684 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:48.457784 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:48.457923 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:48.460415 (+ 2492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:48.462135 (+ 1720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:48.464870 (+ 2735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:48.465872 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:48.466075 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:48.472159 (+ 6084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:48.472164 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:48.472169 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:48.472242 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:48.476799 (+ 4557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:48.478305 (+ 1506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:48.478313 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:48.478319 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:48.478476 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:48.479137 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:48.479147 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:48.479150 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:48.479547 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:48.479738 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:48.481066 (+ 1328us) server_negotiation.cc:299] Negotiation successful
0807 10:49:48.481391 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":410,"threads_started":1}
I20260807 10:49:48.483851 6368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:48.454013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41773 (local address 127.3.73.194:42503)
0807 10:49:48.454988 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:48.455042 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:48.457294 (+ 2252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:48.458339 (+ 1045us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:48.458356 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:48.459233 (+ 877us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:49:48.460116 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:48.460138 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:48.462342 (+ 2204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:48.462350 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:48.464652 (+ 2302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:48.464667 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:48.466522 (+ 1855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:48.466531 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:48.466822 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:48.467887 (+ 1065us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:49:48.467931 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:49:48.471857 (+ 3926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:49:48.477066 (+ 5209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:48.477082 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:48.477107 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:48.478097 (+ 990us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:48.478615 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:48.478620 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:48.478626 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:48.479006 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:48.479734 (+ 728us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:49:48.479748 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:49:48.480412 (+ 664us) client_negotiation.cc:769] Sending connection context
0807 10:49:48.480860 (+ 448us) client_negotiation.cc:240] Negotiation successful
0807 10:49:48.481645 (+ 785us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"thread_start_us":429,"threads_started":1}
I20260807 10:49:48.505043 6334 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.194:41391
I20260807 10:49:48.505079 6538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.194:41391 every 8 connection(s)
I20260807 10:49:48.506520 6334 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
I20260807 10:49:48.508131 6367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:48.490703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:51935 (local address 127.3.73.254:41773)
0807 10:49:48.490958 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:48.490967 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:48.491227 (+ 260us) server_negotiation.cc:407] Connection header received
0807 10:49:48.491490 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:48.491496 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:48.491580 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:48.491707 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:48.493404 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:48.494639 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:48.496656 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:48.497355 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:48.497504 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:48.500865 (+ 3361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:48.500876 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:48.500883 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:48.500938 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:48.504645 (+ 3707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:48.505675 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:48.505683 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:48.505687 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:48.505846 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:48.506329 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:48.506340 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:48.506342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:48.506764 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:48.506946 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:48.507455 (+ 509us) server_negotiation.cc:299] Negotiation successful
0807 10:49:48.507724 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 10:49:48.508319 6495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:48.490268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41773 (local address 127.3.73.194:51935)
0807 10:49:48.491085 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:48.491110 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:48.491315 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:48.491753 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:48.491758 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:48.492504 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:49:48.493171 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:48.493208 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:48.494855 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:48.494860 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:48.496489 (+ 1629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:48.496505 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:48.497646 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:48.497654 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:48.497819 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:48.498875 (+ 1056us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:49:48.498896 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:49:48.500441 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:49:48.504900 (+ 4459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:48.504911 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:48.504916 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:48.505527 (+ 611us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:48.505982 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:48.505988 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:48.505992 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:48.506180 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:48.506906 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:49:48.506911 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:49:48.507137 (+ 226us) client_negotiation.cc:769] Sending connection context
0807 10:49:48.507349 (+ 212us) client_negotiation.cc:240] Negotiation successful
0807 10:49:48.507783 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"thread_start_us":526,"threads_started":1}
I20260807 10:49:48.509586 6539 heartbeater.cc:347] Connected to a master server at 127.3.73.254:41773
I20260807 10:49:48.509843 6539 heartbeater.cc:464] Registering TS with master...
I20260807 10:49:48.510721 6539 heartbeater.cc:511] Master 127.3.73.254:41773 requested a full tablet report, sending...
I20260807 10:49:48.512671 6010 ts_manager.cc:194] Registered new tserver with Master: dd8042b5b9ce4a61a5ca306878f9ab7a (127.3.73.194:41391)
I20260807 10:49:48.513798 6010 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.194@KRBTEST.COM'} at 127.3.73.194:42503
I20260807 10:49:48.516369 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 6334
I20260807 10:49:48.516580 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance
I20260807 10:49:48.517122 3367 external_mini_cluster.cc:1428] Setting key 97f577b199f5040ea36a4fb1992eaa2e
WARNING: no policy specified for kudu/127.3.73.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab.
Entry for principal kudu/127.3.73.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab.
I20260807 10:49:48.576335 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.195:0
--local_ip_for_outbound_sockets=127.3.73.195
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:41773
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:36429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 10:49:48.929579 6329 heartbeater.cc:503] Master 127.3.73.254:41773 was elected leader, sending a full tablet report...
W20260807 10:49:48.932389 6543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:49:48.932731 6543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:49:48.932952 6543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:49:48.940564 6543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:49:48.940758 6543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:49:48.940943 6543 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 10:49:48.941035 6543 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 10:49:48.941145 6543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.195
I20260807 10:49:48.951725 6543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:36429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.73.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.73.254:41773
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.195
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:49:48.953266 6543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:49:48.954691 6543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:49:48.969027 6549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:48.969090 6548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:48.969218 6552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:49:48.970180 6543 server_base.cc:1034] running on GCE node
I20260807 10:49:48.971007 6543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:49:48.972078 6543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:49:48.973285 6543 hybrid_clock.cc:648] HybridClock initialized: now 1786099788973245 us; error 39 us; skew 500 ppm
I20260807 10:49:48.973631 6543 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:49:48 dist-test-slave-x5fp krb5kdc[5938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099788, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:49:48.976972 6543 init.cc:377] Logged in from keytab as kudu/127.3.73.195@KRBTEST.COM (short username kudu)
I20260807 10:49:48.978523 6543 webserver.cc:460] Webserver started at http://127.3.73.195:36821/ using document root <none> and password file <none>
I20260807 10:49:48.979344 6543 fs_manager.cc:359] Metadata directory not provided
I20260807 10:49:48.979454 6543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:49:48.979729 6543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:49:48.983175 6543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-2/data/instance:
uuid: "3519d7d61a6f4a50909174f45b035d9b"
format_stamp: "Formatted at 2026-08-07 10:49:48 on dist-test-slave-x5fp"
server_key: "095cc35f23382eb2dbab42749b1bb541"
server_key_iv: "b4f1841020e67d7192d736a9fca3c447"
server_key_version: "encryptionkey@0"
I20260807 10:49:48.984012 6543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance:
uuid: "3519d7d61a6f4a50909174f45b035d9b"
format_stamp: "Formatted at 2026-08-07 10:49:48 on dist-test-slave-x5fp"
server_key: "095cc35f23382eb2dbab42749b1bb541"
server_key_iv: "b4f1841020e67d7192d736a9fca3c447"
server_key_version: "encryptionkey@0"
I20260807 10:49:48.989909 6543 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260807 10:49:48.993731 6566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:48.995261 6543 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 10:49:48.995410 6543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-2/wal
uuid: "3519d7d61a6f4a50909174f45b035d9b"
format_stamp: "Formatted at 2026-08-07 10:49:48 on dist-test-slave-x5fp"
server_key: "095cc35f23382eb2dbab42749b1bb541"
server_key_iv: "b4f1841020e67d7192d736a9fca3c447"
server_key_version: "encryptionkey@0"
I20260807 10:49:48.995599 6543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:49:49.009447 6543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:49:49.014284 6543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:49:49.014540 6543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:49:49.017443 6543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:49:49.017601 6543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:49.017752 6543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:49:49.017920 6543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Aug 07 10:49:49 dist-test-slave-x5fp krb5kdc[5938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099788, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:49:49.044886 6579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:49.018043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:53639 (local address 127.3.73.254:41773)
0807 10:49:49.018910 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:49.018917 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:49.019975 (+ 1058us) server_negotiation.cc:407] Connection header received
0807 10:49:49.022523 (+ 2548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:49.022529 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:49.022649 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:49.022812 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:49.025801 (+ 2989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.026576 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.029876 (+ 3300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.030370 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.030482 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:49.036221 (+ 5739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:49.036226 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:49.036230 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:49.036268 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:49.040268 (+ 4000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:49.041529 (+ 1261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:49.041538 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:49.041545 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:49.041776 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:49.042307 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:49.042315 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:49.042321 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:49.042785 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:49.042974 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:49.043881 (+ 907us) server_negotiation.cc:299] Negotiation successful
0807 10:49:49.044225 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":726,"thread_start_us":602,"threads_started":1}
I20260807 10:49:49.045809 6580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:49.018781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41773 (local address 127.3.73.195:53639)
0807 10:49:49.019751 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:49.019818 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:49.022206 (+ 2388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:49.023145 (+ 939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:49.023160 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:49.024337 (+ 1177us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:49:49.025545 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:49.025585 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.026757 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.026766 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:49.029677 (+ 2911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:49.029704 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.030658 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.030669 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:49.031040 (+ 371us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:49.031974 (+ 934us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:49:49.032016 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:49:49.035920 (+ 3904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:49:49.040460 (+ 4540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:49.040471 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:49.040499 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:49.041336 (+ 837us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:49.041941 (+ 605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:49.041945 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:49.041948 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:49.042170 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:49.042977 (+ 807us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:49:49.042984 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:49:49.043453 (+ 469us) client_negotiation.cc:769] Sending connection context
0807 10:49:49.043715 (+ 262us) client_negotiation.cc:240] Negotiation successful
0807 10:49:49.044230 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"thread_start_us":443,"threads_started":1}
I20260807 10:49:49.080199 6543 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.195:39467
I20260807 10:49:49.080250 6744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.195:39467 every 8 connection(s)
I20260807 10:49:49.081422 6579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:49.060147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:41867 (local address 127.3.73.254:41773)
0807 10:49:49.060365 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:49.060375 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:49.065598 (+ 5223us) server_negotiation.cc:407] Connection header received
0807 10:49:49.065831 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:49.065840 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:49.065965 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:49.066107 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:49.067423 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.068970 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.070971 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.071646 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.071796 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:49.074781 (+ 2985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:49.074785 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:49.074789 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:49.074832 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:49.078205 (+ 3373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:49.078969 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:49.078975 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:49.078979 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:49.079131 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:49.079735 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:49.079740 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:49.079743 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:49.080070 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:49.080239 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:49.080801 (+ 562us) server_negotiation.cc:299] Negotiation successful
0807 10:49:49.081050 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 10:49:49.081521 6693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:49.060144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41773 (local address 127.3.73.195:41867)
0807 10:49:49.065461 (+ 5317us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:49.065481 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:49.065676 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:49.066162 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:49.066166 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:49.066829 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:49:49.067245 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:49.067258 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.069165 (+ 1907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.069176 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:49.070779 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:49.070812 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.071897 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.071904 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:49.072091 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:49.073054 (+ 963us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:49:49.073086 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:49:49.074496 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:49:49.078357 (+ 3861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:49.078362 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:49.078367 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:49.078812 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:49.079314 (+ 502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:49.079322 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:49.079329 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:49.079554 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:49.080202 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:49:49.080209 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:49:49.080467 (+ 258us) client_negotiation.cc:769] Sending connection context
0807 10:49:49.080647 (+ 180us) client_negotiation.cc:240] Negotiation successful
0807 10:49:49.081050 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5138,"thread_start_us":4888,"threads_started":1}
I20260807 10:49:49.081867 6543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
I20260807 10:49:49.083465 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 6543
I20260807 10:49:49.083647 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance
I20260807 10:49:49.084245 3367 external_mini_cluster.cc:1428] Setting key 2376e97509120498f181685eb1319f6b
I20260807 10:49:49.084429 6745 heartbeater.cc:347] Connected to a master server at 127.3.73.254:41773
I20260807 10:49:49.085935 6745 heartbeater.cc:464] Registering TS with master...
I20260807 10:49:49.086817 6745 heartbeater.cc:511] Master 127.3.73.254:41773 requested a full tablet report, sending...
I20260807 10:49:49.088444 6010 ts_manager.cc:194] Registered new tserver with Master: 3519d7d61a6f4a50909174f45b035d9b (127.3.73.195:39467)
I20260807 10:49:49.089362 6010 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.195@KRBTEST.COM'} at 127.3.73.195:53639
I20260807 10:49:49.100840 3367 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 10:49:49 dist-test-slave-x5fp krb5kdc[5938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 10:49:49 dist-test-slave-x5fp krb5kdc[5938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099789, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:49:49.165956 6579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:49.149497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46874 (local address 127.3.73.254:41773)
0807 10:49:49.149752 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:49.149757 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:49.150088 (+ 331us) server_negotiation.cc:407] Connection header received
0807 10:49:49.150279 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:49.150284 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:49.150364 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:49.150472 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:49.152187 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.153150 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.154537 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.154989 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.155103 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:49.159391 (+ 4288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:49.159398 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:49.159402 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:49.159450 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:49.162578 (+ 3128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:49.163478 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:49.163484 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:49.163486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:49.163635 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:49.164030 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:49.164034 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:49.164036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:49.164368 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:49.164602 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:49.165110 (+ 508us) server_negotiation.cc:299] Negotiation successful
0807 10:49:49.165524 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 10:49:49.171068 6010 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46874:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 10:49:49.178117 6010 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 10:49:49.203866 6768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:49.192435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:41391 (local address 127.0.0.1:53254)
0807 10:49:49.193404 (+ 969us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:49.193457 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:49.193631 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:49.195006 (+ 1375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:49.195014 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:49.195059 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:49:49.195583 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:49.195597 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.197757 (+ 2160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.197763 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:49.200527 (+ 2764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:49.200543 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.202683 (+ 2140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.202690 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:49.202950 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:49.203015 (+ 65us) client_negotiation.cc:769] Sending connection context
0807 10:49:49.203135 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 10:49:49.203392 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"mutex_wait_us":24,"thread_start_us":1876,"threads_started":1}
I20260807 10:49:49.204165 6770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:49.193269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53254 (local address 127.3.73.194:41391)
0807 10:49:49.194282 (+ 1013us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:49.194293 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:49.194343 (+ 50us) server_negotiation.cc:407] Connection header received
0807 10:49:49.194456 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:49.194465 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:49.194754 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:49.195786 (+ 1032us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:49:49.196020 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.197601 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.200754 (+ 3153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.202410 (+ 1656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.203405 (+ 995us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:49.203614 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:49.203688 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 10:49:49.203884 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":804,"thread_start_us":649,"threads_started":1}
I20260807 10:49:49.204900 6763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:49.192570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:44797 (local address 127.0.0.1:42858)
0807 10:49:49.194580 (+ 2010us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:49.194606 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:49.194725 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:49.195699 (+ 974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:49.195706 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:49.195738 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:49:49.196202 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:49.196214 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.198207 (+ 1993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.198212 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:49.200771 (+ 2559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:49.200784 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.202985 (+ 2201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.202991 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:49.203186 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:49.203215 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 10:49:49.203292 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 10:49:49.204469 (+ 1177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1861,"mutex_wait_us":85,"thread_start_us":745,"threads_started":1}
I20260807 10:49:49.204964 6769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:49.193184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42858 (local address 127.3.73.193:44797)
0807 10:49:49.194241 (+ 1057us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:49.194260 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:49.194870 (+ 610us) server_negotiation.cc:407] Connection header received
0807 10:49:49.194986 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:49.194994 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:49.195238 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:49.195454 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:49:49.196407 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.198023 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.200971 (+ 2948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.202410 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.202576 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:49.202832 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:49.204454 (+ 1622us) server_negotiation.cc:299] Negotiation successful
0807 10:49:49.204644 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":856,"thread_start_us":761,"threads_started":1}
I20260807 10:49:49.209671 6220 tablet_service.cc:1467] Processing CreateTablet for tablet 3a82b120b6e640a58a589c1c96abf3ae (DEFAULT_TABLE table=test-table [id=d9d4749e46554e759549f0482a9bc377]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:49:49.211398 6220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a82b120b6e640a58a589c1c96abf3ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:49:49.211382 6435 tablet_service.cc:1467] Processing CreateTablet for tablet 3a82b120b6e640a58a589c1c96abf3ae (DEFAULT_TABLE table=test-table [id=d9d4749e46554e759549f0482a9bc377]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:49:49.213032 6435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a82b120b6e640a58a589c1c96abf3ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:49:49.214352 6768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:49.194747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:39467 (local address 127.0.0.1:38242)
0807 10:49:49.204121 (+ 9374us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:49.204144 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:49.204258 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:49.207271 (+ 3013us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:49.207276 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:49.207312 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:49:49.207502 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:49.207515 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.209417 (+ 1902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.209423 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:49.211347 (+ 1924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:49.211363 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.213549 (+ 2186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.213555 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:49.213765 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:49.213802 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 10:49:49.213891 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 10:49:49.214040 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9275,"thread_start_us":12557,"threads_started":1}
I20260807 10:49:49.215247 6772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:49.195079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38242 (local address 127.3.73.195:39467)
0807 10:49:49.206299 (+ 11220us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:49.206310 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:49.206346 (+ 36us) server_negotiation.cc:407] Connection header received
0807 10:49:49.206460 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:49.206469 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:49.206743 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:49.207200 (+ 457us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:49:49.207729 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.209223 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.211551 (+ 2328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.213087 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.213209 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:49.213450 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:49.214746 (+ 1296us) server_negotiation.cc:299] Negotiation successful
0807 10:49:49.214925 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11027,"thread_start_us":10743,"threads_started":1}
I20260807 10:49:49.219132 6640 tablet_service.cc:1467] Processing CreateTablet for tablet 3a82b120b6e640a58a589c1c96abf3ae (DEFAULT_TABLE table=test-table [id=d9d4749e46554e759549f0482a9bc377]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:49:49.220909 6640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a82b120b6e640a58a589c1c96abf3ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:49:49.232575 6773 tablet_bootstrap.cc:492] T 3a82b120b6e640a58a589c1c96abf3ae P dd8042b5b9ce4a61a5ca306878f9ab7a: Bootstrap starting.
I20260807 10:49:49.234849 6774 tablet_bootstrap.cc:492] T 3a82b120b6e640a58a589c1c96abf3ae P ea7e70332cf64545889d381b59a7d558: Bootstrap starting.
I20260807 10:49:49.236626 6773 tablet_bootstrap.cc:654] T 3a82b120b6e640a58a589c1c96abf3ae P dd8042b5b9ce4a61a5ca306878f9ab7a: Neither blocks nor log segments found. Creating new log.
I20260807 10:49:49.237739 6773 log.cc:824] T 3a82b120b6e640a58a589c1c96abf3ae P dd8042b5b9ce4a61a5ca306878f9ab7a: Log is configured to *not* fsync() on all Append() calls
I20260807 10:49:49.238920 6774 tablet_bootstrap.cc:654] T 3a82b120b6e640a58a589c1c96abf3ae P ea7e70332cf64545889d381b59a7d558: Neither blocks nor log segments found. Creating new log.
I20260807 10:49:49.240329 6774 log.cc:824] T 3a82b120b6e640a58a589c1c96abf3ae P ea7e70332cf64545889d381b59a7d558: Log is configured to *not* fsync() on all Append() calls
I20260807 10:49:49.241243 6773 tablet_bootstrap.cc:492] T 3a82b120b6e640a58a589c1c96abf3ae P dd8042b5b9ce4a61a5ca306878f9ab7a: No bootstrap required, opened a new log
I20260807 10:49:49.241591 6773 ts_tablet_manager.cc:1397] T 3a82b120b6e640a58a589c1c96abf3ae P dd8042b5b9ce4a61a5ca306878f9ab7a: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.002s
I20260807 10:49:49.242372 6781 tablet_bootstrap.cc:492] T 3a82b120b6e640a58a589c1c96abf3ae P 3519d7d61a6f4a50909174f45b035d9b: Bootstrap starting.
I20260807 10:49:49.244336 6774 tablet_bootstrap.cc:492] T 3a82b120b6e640a58a589c1c96abf3ae P ea7e70332cf64545889d381b59a7d558: No bootstrap required, opened a new log
I20260807 10:49:49.244623 6774 ts_tablet_manager.cc:1397] T 3a82b120b6e640a58a589c1c96abf3ae P ea7e70332cf64545889d381b59a7d558: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260807 10:49:49.246106 6781 tablet_bootstrap.cc:654] T 3a82b120b6e640a58a589c1c96abf3ae P 3519d7d61a6f4a50909174f45b035d9b: Neither blocks nor log segments found. Creating new log.
I20260807 10:49:49.247364 6781 log.cc:824] T 3a82b120b6e640a58a589c1c96abf3ae P 3519d7d61a6f4a50909174f45b035d9b: Log is configured to *not* fsync() on all Append() calls
I20260807 10:49:49.250896 6781 tablet_bootstrap.cc:492] T 3a82b120b6e640a58a589c1c96abf3ae P 3519d7d61a6f4a50909174f45b035d9b: No bootstrap required, opened a new log
I20260807 10:49:49.251184 6781 ts_tablet_manager.cc:1397] T 3a82b120b6e640a58a589c1c96abf3ae P 3519d7d61a6f4a50909174f45b035d9b: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 10:49:49.252893 6773 raft_consensus.cc:348] T 3a82b120b6e640a58a589c1c96abf3ae P dd8042b5b9ce4a61a5ca306878f9ab7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:49.253364 6773 raft_consensus.cc:374] T 3a82b120b6e640a58a589c1c96abf3ae P dd8042b5b9ce4a61a5ca306878f9ab7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:49:49.253480 6773 raft_consensus.cc:729] T 3a82b120b6e640a58a589c1c96abf3ae P dd8042b5b9ce4a61a5ca306878f9ab7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd8042b5b9ce4a61a5ca306878f9ab7a, State: Initialized, Role: FOLLOWER
I20260807 10:49:49.254169 6773 consensus_queue.cc:260] T 3a82b120b6e640a58a589c1c96abf3ae P dd8042b5b9ce4a61a5ca306878f9ab7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:49.255437 6539 heartbeater.cc:503] Master 127.3.73.254:41773 was elected leader, sending a full tablet report...
I20260807 10:49:49.255715 6774 raft_consensus.cc:348] T 3a82b120b6e640a58a589c1c96abf3ae P ea7e70332cf64545889d381b59a7d558 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:49.256253 6774 raft_consensus.cc:374] T 3a82b120b6e640a58a589c1c96abf3ae P ea7e70332cf64545889d381b59a7d558 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:49:49.256407 6774 raft_consensus.cc:729] T 3a82b120b6e640a58a589c1c96abf3ae P ea7e70332cf64545889d381b59a7d558 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea7e70332cf64545889d381b59a7d558, State: Initialized, Role: FOLLOWER
I20260807 10:49:49.256577 6773 ts_tablet_manager.cc:1428] T 3a82b120b6e640a58a589c1c96abf3ae P dd8042b5b9ce4a61a5ca306878f9ab7a: Time spent starting tablet: real 0.015s user 0.012s sys 0.002s
I20260807 10:49:49.257154 6774 consensus_queue.cc:260] T 3a82b120b6e640a58a589c1c96abf3ae P ea7e70332cf64545889d381b59a7d558 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:49.259336 6774 ts_tablet_manager.cc:1428] T 3a82b120b6e640a58a589c1c96abf3ae P ea7e70332cf64545889d381b59a7d558: Time spent starting tablet: real 0.015s user 0.008s sys 0.005s
W20260807 10:49:49.260195 6540 tablet.cc:2369] T 3a82b120b6e640a58a589c1c96abf3ae P dd8042b5b9ce4a61a5ca306878f9ab7a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:49:49.264313 6781 raft_consensus.cc:348] T 3a82b120b6e640a58a589c1c96abf3ae P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:49.264861 6781 raft_consensus.cc:374] T 3a82b120b6e640a58a589c1c96abf3ae P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:49:49.264990 6781 raft_consensus.cc:729] T 3a82b120b6e640a58a589c1c96abf3ae P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3519d7d61a6f4a50909174f45b035d9b, State: Initialized, Role: FOLLOWER
I20260807 10:49:49.265689 6781 consensus_queue.cc:260] T 3a82b120b6e640a58a589c1c96abf3ae P 3519d7d61a6f4a50909174f45b035d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:49.266907 6745 heartbeater.cc:503] Master 127.3.73.254:41773 was elected leader, sending a full tablet report...
I20260807 10:49:49.268059 6781 ts_tablet_manager.cc:1428] T 3a82b120b6e640a58a589c1c96abf3ae P 3519d7d61a6f4a50909174f45b035d9b: Time spent starting tablet: real 0.017s user 0.008s sys 0.007s
W20260807 10:49:49.334211 6746 tablet.cc:2369] T 3a82b120b6e640a58a589c1c96abf3ae P 3519d7d61a6f4a50909174f45b035d9b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 10:49:49.418918 6331 tablet.cc:2369] T 3a82b120b6e640a58a589c1c96abf3ae P ea7e70332cf64545889d381b59a7d558: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:49:49.644567 6784 raft_consensus.cc:482] T 3a82b120b6e640a58a589c1c96abf3ae P dd8042b5b9ce4a61a5ca306878f9ab7a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 10:49:49.645159 6784 raft_consensus.cc:504] T 3a82b120b6e640a58a589c1c96abf3ae P dd8042b5b9ce4a61a5ca306878f9ab7a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:49.647058 6787 raft_consensus.cc:482] T 3a82b120b6e640a58a589c1c96abf3ae P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 10:49:49.647384 6787 raft_consensus.cc:504] T 3a82b120b6e640a58a589c1c96abf3ae P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:49.649756 6787 leader_election.cc:288] T 3a82b120b6e640a58a589c1c96abf3ae P 3519d7d61a6f4a50909174f45b035d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd8042b5b9ce4a61a5ca306878f9ab7a (127.3.73.194:41391), ea7e70332cf64545889d381b59a7d558 (127.3.73.193:44797)
I20260807 10:49:49.650442 6784 leader_election.cc:288] T 3a82b120b6e640a58a589c1c96abf3ae P dd8042b5b9ce4a61a5ca306878f9ab7a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea7e70332cf64545889d381b59a7d558 (127.3.73.193:44797), 3519d7d61a6f4a50909174f45b035d9b (127.3.73.195:39467)
I20260807 10:49:49.662611 6769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:49.648011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:50535 (local address 127.3.73.193:44797)
0807 10:49:49.648252 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:49.648260 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:49.650659 (+ 2399us) server_negotiation.cc:407] Connection header received
0807 10:49:49.651071 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:49.651082 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:49.651239 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:49.651324 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:49:49.652594 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.654139 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.659651 (+ 5512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.661227 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.661375 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:49.661460 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:49.662140 (+ 680us) server_negotiation.cc:299] Negotiation successful
0807 10:49:49.662299 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 10:49:49.663699 6788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:49.647772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:44797 (local address 127.3.73.194:50535)
0807 10:49:49.650526 (+ 2754us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:49.650553 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:49.650835 (+ 282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:49.651622 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:49.651630 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:49.651667 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:49:49.652261 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:49.652353 (+ 92us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.654430 (+ 2077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.654440 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:49.659376 (+ 4936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:49.659404 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.661698 (+ 2294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.661705 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:49.661911 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:49.661953 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 10:49:49.663083 (+ 1130us) client_negotiation.cc:240] Negotiation successful
0807 10:49:49.663267 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2583,"thread_start_us":2387,"threads_started":1}
I20260807 10:49:49.664974 6264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a82b120b6e640a58a589c1c96abf3ae" candidate_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea7e70332cf64545889d381b59a7d558" is_pre_election: true
I20260807 10:49:49.665902 6264 raft_consensus.cc:2454] T 3a82b120b6e640a58a589c1c96abf3ae P ea7e70332cf64545889d381b59a7d558 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd8042b5b9ce4a61a5ca306878f9ab7a in term 0.
I20260807 10:49:49.666904 6358 leader_election.cc:302] T 3a82b120b6e640a58a589c1c96abf3ae P dd8042b5b9ce4a61a5ca306878f9ab7a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd8042b5b9ce4a61a5ca306878f9ab7a, ea7e70332cf64545889d381b59a7d558; no voters:
I20260807 10:49:49.667431 6784 raft_consensus.cc:2781] T 3a82b120b6e640a58a589c1c96abf3ae P dd8042b5b9ce4a61a5ca306878f9ab7a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 10:49:49.667595 6784 raft_consensus.cc:482] T 3a82b120b6e640a58a589c1c96abf3ae P dd8042b5b9ce4a61a5ca306878f9ab7a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 10:49:49.667704 6784 raft_consensus.cc:3037] T 3a82b120b6e640a58a589c1c96abf3ae P dd8042b5b9ce4a61a5ca306878f9ab7a [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:49:49.671654 6784 raft_consensus.cc:504] T 3a82b120b6e640a58a589c1c96abf3ae P dd8042b5b9ce4a61a5ca306878f9ab7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:49.673202 6264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a82b120b6e640a58a589c1c96abf3ae" candidate_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea7e70332cf64545889d381b59a7d558"
I20260807 10:49:49.673476 6264 raft_consensus.cc:3037] T 3a82b120b6e640a58a589c1c96abf3ae P ea7e70332cf64545889d381b59a7d558 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:49:49.677179 6770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:49.650902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:34371 (local address 127.3.73.194:41391)
0807 10:49:49.651136 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:49.651144 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:49.660327 (+ 9183us) server_negotiation.cc:407] Connection header received
0807 10:49:49.661313 (+ 986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:49.661318 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:49.661416 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:49.661480 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:49:49.663266 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.664476 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.672159 (+ 7683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.673427 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.673587 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:49.673672 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:49.676634 (+ 2962us) server_negotiation.cc:299] Negotiation successful
0807 10:49:49.676818 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 10:49:49.677122 6264 raft_consensus.cc:2454] T 3a82b120b6e640a58a589c1c96abf3ae P ea7e70332cf64545889d381b59a7d558 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd8042b5b9ce4a61a5ca306878f9ab7a in term 1.
I20260807 10:49:49.677637 6358 leader_election.cc:302] T 3a82b120b6e640a58a589c1c96abf3ae P dd8042b5b9ce4a61a5ca306878f9ab7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd8042b5b9ce4a61a5ca306878f9ab7a, ea7e70332cf64545889d381b59a7d558; no voters:
I20260807 10:49:49.677966 6791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:49.650000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:41391 (local address 127.3.73.195:34371)
0807 10:49:49.660223 (+ 10223us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:49.660243 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:49.660870 (+ 627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:49.661707 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:49.661711 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:49.661741 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:49:49.661983 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:49.661998 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.664740 (+ 2742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.664749 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:49.671922 (+ 7173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:49.671941 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.673856 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.673864 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:49.674094 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:49.674132 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 10:49:49.677459 (+ 3327us) client_negotiation.cc:240] Negotiation successful
0807 10:49:49.677623 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10094,"threads_started":1}
I20260807 10:49:49.678786 6784 leader_election.cc:288] T 3a82b120b6e640a58a589c1c96abf3ae P dd8042b5b9ce4a61a5ca306878f9ab7a [CANDIDATE]: Term 1 election: Requested vote from peers ea7e70332cf64545889d381b59a7d558 (127.3.73.193:44797), 3519d7d61a6f4a50909174f45b035d9b (127.3.73.195:39467)
I20260807 10:49:49.678928 6772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:49.648909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:56493 (local address 127.3.73.195:39467)
0807 10:49:49.649121 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:49.649129 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:49.659818 (+ 10689us) server_negotiation.cc:407] Connection header received
0807 10:49:49.662173 (+ 2355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:49.662178 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:49.662246 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:49.662300 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:49:49.666674 (+ 4374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.667746 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.674544 (+ 6798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.675803 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.675940 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:49.676029 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:49.678534 (+ 2505us) server_negotiation.cc:299] Negotiation successful
0807 10:49:49.678676 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 10:49:49.679025 6784 raft_consensus.cc:2781] T 3a82b120b6e640a58a589c1c96abf3ae P dd8042b5b9ce4a61a5ca306878f9ab7a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:49:49.679201 6784 raft_consensus.cc:686] T 3a82b120b6e640a58a589c1c96abf3ae P dd8042b5b9ce4a61a5ca306878f9ab7a [term 1 LEADER]: Becoming Leader. State: Replica: dd8042b5b9ce4a61a5ca306878f9ab7a, State: Running, Role: LEADER
I20260807 10:49:49.679767 6789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:49.648405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:39467 (local address 127.3.73.194:56493)
0807 10:49:49.659681 (+ 11276us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:49.659703 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:49.661122 (+ 1419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:49.662581 (+ 1459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:49.662588 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:49.662621 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:49:49.663083 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:49.663097 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.667953 (+ 4856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.667959 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:49.674369 (+ 6410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:49.674384 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.676252 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.676259 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:49.676445 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:49.676479 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 10:49:49.679247 (+ 2768us) client_negotiation.cc:240] Negotiation successful
0807 10:49:49.679412 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11122,"thread_start_us":11923,"threads_started":1}
I20260807 10:49:49.679845 6784 consensus_queue.cc:237] T 3a82b120b6e640a58a589c1c96abf3ae P dd8042b5b9ce4a61a5ca306878f9ab7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:49.681272 6671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a82b120b6e640a58a589c1c96abf3ae" candidate_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3519d7d61a6f4a50909174f45b035d9b" is_pre_election: true
I20260807 10:49:49.681778 6671 raft_consensus.cc:2454] T 3a82b120b6e640a58a589c1c96abf3ae P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd8042b5b9ce4a61a5ca306878f9ab7a in term 0.
I20260807 10:49:49.682420 6670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a82b120b6e640a58a589c1c96abf3ae" candidate_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3519d7d61a6f4a50909174f45b035d9b"
I20260807 10:49:49.682828 6670 raft_consensus.cc:3037] T 3a82b120b6e640a58a589c1c96abf3ae P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:49:49.688409 6670 raft_consensus.cc:2454] T 3a82b120b6e640a58a589c1c96abf3ae P 3519d7d61a6f4a50909174f45b035d9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd8042b5b9ce4a61a5ca306878f9ab7a in term 1.
I20260807 10:49:49.689101 6769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:49.651558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:55393 (local address 127.3.73.193:44797)
0807 10:49:49.662931 (+ 11373us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:49.662938 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:49.678393 (+ 15455us) server_negotiation.cc:407] Connection header received
0807 10:49:49.680384 (+ 1991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:49.680394 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:49.680529 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:49.680611 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:49:49.681404 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.682676 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.685457 (+ 2781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.687438 (+ 1981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.687627 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:49.687743 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:49.688447 (+ 704us) server_negotiation.cc:299] Negotiation successful
0807 10:49:49.688646 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11237,"thread_start_us":27635,"threads_started":1}
I20260807 10:49:49.690024 6791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:49.650776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:44797 (local address 127.3.73.195:55393)
0807 10:49:49.678257 (+ 27481us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:49.678284 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:49.680169 (+ 1885us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:49.680902 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:49.680910 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:49.680952 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:49:49.681180 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:49.681195 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.682900 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.682906 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:49.685172 (+ 2266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:49.685195 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.687918 (+ 2723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.687927 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:49.688200 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:49.688252 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 10:49:49.689415 (+ 1163us) client_negotiation.cc:240] Negotiation successful
0807 10:49:49.689638 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27329,"thread_start_us":9198,"threads_started":1}
I20260807 10:49:49.690459 6463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a82b120b6e640a58a589c1c96abf3ae" candidate_uuid: "3519d7d61a6f4a50909174f45b035d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" is_pre_election: true
I20260807 10:49:49.690881 6264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a82b120b6e640a58a589c1c96abf3ae" candidate_uuid: "3519d7d61a6f4a50909174f45b035d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea7e70332cf64545889d381b59a7d558" is_pre_election: true
I20260807 10:49:49.691349 6264 raft_consensus.cc:2379] T 3a82b120b6e640a58a589c1c96abf3ae P ea7e70332cf64545889d381b59a7d558 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3519d7d61a6f4a50909174f45b035d9b in current term 1: Already voted for candidate dd8042b5b9ce4a61a5ca306878f9ab7a in this term.
I20260807 10:49:49.692585 6567 leader_election.cc:302] T 3a82b120b6e640a58a589c1c96abf3ae P 3519d7d61a6f4a50909174f45b035d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3519d7d61a6f4a50909174f45b035d9b; no voters: dd8042b5b9ce4a61a5ca306878f9ab7a, ea7e70332cf64545889d381b59a7d558
I20260807 10:49:49.693094 6787 raft_consensus.cc:2726] T 3a82b120b6e640a58a589c1c96abf3ae P 3519d7d61a6f4a50909174f45b035d9b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 10:49:49.695612 6010 catalog_manager.cc:5495] T 3a82b120b6e640a58a589c1c96abf3ae P dd8042b5b9ce4a61a5ca306878f9ab7a reported cstate change: term changed from 0 to 1, leader changed from <none> to dd8042b5b9ce4a61a5ca306878f9ab7a (127.3.73.194). New cstate: current_term: 1 leader_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } health_report { overall_health: UNKNOWN } } }
Aug 07 10:49:49 dist-test-slave-x5fp krb5kdc[5938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099787, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:49:49.839488 6812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:49.823639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46882 (local address 127.3.73.254:41773)
0807 10:49:49.824774 (+ 1135us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:49.824782 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:49.824810 (+ 28us) server_negotiation.cc:407] Connection header received
0807 10:49:49.824868 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:49.824872 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:49.824932 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:49.825057 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:49.826304 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.827079 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.828432 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.829037 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.829182 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:49.832705 (+ 3523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:49.832710 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:49.832757 (+ 47us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:49.832796 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:49.835755 (+ 2959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:49.837021 (+ 1266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:49.837026 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:49.837029 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:49.837172 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:49.837766 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:49.837769 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:49.837771 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:49.838062 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:49.838173 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:49.838984 (+ 811us) server_negotiation.cc:299] Negotiation successful
0807 10:49:49.839181 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":925,"thread_start_us":721,"threads_started":1}
I20260807 10:49:49.840152 6810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:49.823552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41773 (local address 127.0.0.1:46882)
0807 10:49:49.824358 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:49.824391 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:49.824586 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:49.825084 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:49.825089 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:49.825860 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:49:49.826118 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:49.826129 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.827221 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.827226 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:49.828284 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:49.828296 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:49.829205 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:49.829211 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:49.829404 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:49.830136 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:49:49.830155 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:49:49.832482 (+ 2327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:49:49.835932 (+ 3450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:49.835950 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:49.835963 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:49.836707 (+ 744us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:49.837335 (+ 628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:49.837344 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:49.837352 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:49.837593 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:49.838302 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:49:49.838314 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:49:49.838617 (+ 303us) client_negotiation.cc:769] Sending connection context
0807 10:49:49.838888 (+ 271us) client_negotiation.cc:240] Negotiation successful
0807 10:49:49.839432 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"mutex_wait_us":116,"thread_start_us":434,"threads_started":1}
I20260807 10:49:49.846477 6010 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:46882:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 10:49:49.847544 6010 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 10:49:49.856161 6220 tablet_service.cc:1467] Processing CreateTablet for tablet 501dceadcab94f87a23022c10d5e2c6a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b8ce6ec2b8b24ef8992328b48908e529]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 10:49:49.857236 6220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 501dceadcab94f87a23022c10d5e2c6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:49:49.857136 6640 tablet_service.cc:1467] Processing CreateTablet for tablet 501dceadcab94f87a23022c10d5e2c6a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b8ce6ec2b8b24ef8992328b48908e529]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 10:49:49.858050 6640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 501dceadcab94f87a23022c10d5e2c6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:49:49.861822 6435 tablet_service.cc:1467] Processing CreateTablet for tablet 501dceadcab94f87a23022c10d5e2c6a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b8ce6ec2b8b24ef8992328b48908e529]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 10:49:49.863147 6435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 501dceadcab94f87a23022c10d5e2c6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:49:49.871754 6813 tablet_bootstrap.cc:492] T 501dceadcab94f87a23022c10d5e2c6a P ea7e70332cf64545889d381b59a7d558: Bootstrap starting.
I20260807 10:49:49.874389 6813 tablet_bootstrap.cc:654] T 501dceadcab94f87a23022c10d5e2c6a P ea7e70332cf64545889d381b59a7d558: Neither blocks nor log segments found. Creating new log.
I20260807 10:49:49.877756 6813 tablet_bootstrap.cc:492] T 501dceadcab94f87a23022c10d5e2c6a P ea7e70332cf64545889d381b59a7d558: No bootstrap required, opened a new log
I20260807 10:49:49.877990 6813 ts_tablet_manager.cc:1397] T 501dceadcab94f87a23022c10d5e2c6a P ea7e70332cf64545889d381b59a7d558: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 10:49:49.879088 6813 raft_consensus.cc:348] T 501dceadcab94f87a23022c10d5e2c6a P ea7e70332cf64545889d381b59a7d558 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:49.879441 6813 raft_consensus.cc:374] T 501dceadcab94f87a23022c10d5e2c6a P ea7e70332cf64545889d381b59a7d558 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:49:49.879557 6813 raft_consensus.cc:729] T 501dceadcab94f87a23022c10d5e2c6a P ea7e70332cf64545889d381b59a7d558 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea7e70332cf64545889d381b59a7d558, State: Initialized, Role: FOLLOWER
I20260807 10:49:49.879860 6813 consensus_queue.cc:260] T 501dceadcab94f87a23022c10d5e2c6a P ea7e70332cf64545889d381b59a7d558 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:49.883190 6815 tablet_bootstrap.cc:492] T 501dceadcab94f87a23022c10d5e2c6a P 3519d7d61a6f4a50909174f45b035d9b: Bootstrap starting.
I20260807 10:49:49.883751 6818 tablet_bootstrap.cc:492] T 501dceadcab94f87a23022c10d5e2c6a P dd8042b5b9ce4a61a5ca306878f9ab7a: Bootstrap starting.
I20260807 10:49:49.886492 6818 tablet_bootstrap.cc:654] T 501dceadcab94f87a23022c10d5e2c6a P dd8042b5b9ce4a61a5ca306878f9ab7a: Neither blocks nor log segments found. Creating new log.
I20260807 10:49:49.887112 6815 tablet_bootstrap.cc:654] T 501dceadcab94f87a23022c10d5e2c6a P 3519d7d61a6f4a50909174f45b035d9b: Neither blocks nor log segments found. Creating new log.
I20260807 10:49:49.896128 6818 tablet_bootstrap.cc:492] T 501dceadcab94f87a23022c10d5e2c6a P dd8042b5b9ce4a61a5ca306878f9ab7a: No bootstrap required, opened a new log
I20260807 10:49:49.896401 6818 ts_tablet_manager.cc:1397] T 501dceadcab94f87a23022c10d5e2c6a P dd8042b5b9ce4a61a5ca306878f9ab7a: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.007s
I20260807 10:49:49.897652 6818 raft_consensus.cc:348] T 501dceadcab94f87a23022c10d5e2c6a P dd8042b5b9ce4a61a5ca306878f9ab7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:49.898089 6818 raft_consensus.cc:374] T 501dceadcab94f87a23022c10d5e2c6a P dd8042b5b9ce4a61a5ca306878f9ab7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:49:49.898221 6818 raft_consensus.cc:729] T 501dceadcab94f87a23022c10d5e2c6a P dd8042b5b9ce4a61a5ca306878f9ab7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd8042b5b9ce4a61a5ca306878f9ab7a, State: Initialized, Role: FOLLOWER
I20260807 10:49:49.898600 6818 consensus_queue.cc:260] T 501dceadcab94f87a23022c10d5e2c6a P dd8042b5b9ce4a61a5ca306878f9ab7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:49.899015 6815 tablet_bootstrap.cc:492] T 501dceadcab94f87a23022c10d5e2c6a P 3519d7d61a6f4a50909174f45b035d9b: No bootstrap required, opened a new log
I20260807 10:49:49.899266 6815 ts_tablet_manager.cc:1397] T 501dceadcab94f87a23022c10d5e2c6a P 3519d7d61a6f4a50909174f45b035d9b: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.008s
I20260807 10:49:49.899209 6784 tablet_replica.cc:439] T 501dceadcab94f87a23022c10d5e2c6a P dd8042b5b9ce4a61a5ca306878f9ab7a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } } }
I20260807 10:49:49.899483 6818 ts_tablet_manager.cc:1428] T 501dceadcab94f87a23022c10d5e2c6a P dd8042b5b9ce4a61a5ca306878f9ab7a: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260807 10:49:49.899544 6784 tablet_replica.cc:442] T 501dceadcab94f87a23022c10d5e2c6a P dd8042b5b9ce4a61a5ca306878f9ab7a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 10:49:49.900584 6815 raft_consensus.cc:348] T 501dceadcab94f87a23022c10d5e2c6a P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:49.901111 6815 raft_consensus.cc:374] T 501dceadcab94f87a23022c10d5e2c6a P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:49:49.901271 6815 raft_consensus.cc:729] T 501dceadcab94f87a23022c10d5e2c6a P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3519d7d61a6f4a50909174f45b035d9b, State: Initialized, Role: FOLLOWER
I20260807 10:49:49.901408 6813 ts_tablet_manager.cc:1428] T 501dceadcab94f87a23022c10d5e2c6a P ea7e70332cf64545889d381b59a7d558: Time spent starting tablet: real 0.023s user 0.012s sys 0.010s
I20260807 10:49:49.901621 6815 consensus_queue.cc:260] T 501dceadcab94f87a23022c10d5e2c6a P 3519d7d61a6f4a50909174f45b035d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:49.902369 6815 ts_tablet_manager.cc:1428] T 501dceadcab94f87a23022c10d5e2c6a P 3519d7d61a6f4a50909174f45b035d9b: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260807 10:49:49.902096 6819 tablet_replica.cc:439] T 501dceadcab94f87a23022c10d5e2c6a P ea7e70332cf64545889d381b59a7d558: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } } }
I20260807 10:49:49.902236 6787 tablet_replica.cc:439] T 501dceadcab94f87a23022c10d5e2c6a P 3519d7d61a6f4a50909174f45b035d9b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } } }
I20260807 10:49:49.902514 6819 tablet_replica.cc:442] T 501dceadcab94f87a23022c10d5e2c6a P ea7e70332cf64545889d381b59a7d558: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 10:49:49.902587 6787 tablet_replica.cc:442] T 501dceadcab94f87a23022c10d5e2c6a P 3519d7d61a6f4a50909174f45b035d9b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 10:49:49.938108 6787 raft_consensus.cc:482] T 501dceadcab94f87a23022c10d5e2c6a P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 10:49:49.938356 6787 raft_consensus.cc:504] T 501dceadcab94f87a23022c10d5e2c6a P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:49.939074 6787 leader_election.cc:288] T 501dceadcab94f87a23022c10d5e2c6a P 3519d7d61a6f4a50909174f45b035d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea7e70332cf64545889d381b59a7d558 (127.3.73.193:44797), dd8042b5b9ce4a61a5ca306878f9ab7a (127.3.73.194:41391)
I20260807 10:49:49.939633 6264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "501dceadcab94f87a23022c10d5e2c6a" candidate_uuid: "3519d7d61a6f4a50909174f45b035d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea7e70332cf64545889d381b59a7d558" is_pre_election: true
I20260807 10:49:49.939849 6463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "501dceadcab94f87a23022c10d5e2c6a" candidate_uuid: "3519d7d61a6f4a50909174f45b035d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" is_pre_election: true
I20260807 10:49:49.939949 6264 raft_consensus.cc:2454] T 501dceadcab94f87a23022c10d5e2c6a P ea7e70332cf64545889d381b59a7d558 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3519d7d61a6f4a50909174f45b035d9b in term 0.
I20260807 10:49:49.940197 6463 raft_consensus.cc:2454] T 501dceadcab94f87a23022c10d5e2c6a P dd8042b5b9ce4a61a5ca306878f9ab7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3519d7d61a6f4a50909174f45b035d9b in term 0.
I20260807 10:49:49.940511 6570 leader_election.cc:302] T 501dceadcab94f87a23022c10d5e2c6a P 3519d7d61a6f4a50909174f45b035d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3519d7d61a6f4a50909174f45b035d9b, ea7e70332cf64545889d381b59a7d558; no voters:
I20260807 10:49:49.940886 6787 raft_consensus.cc:2781] T 501dceadcab94f87a23022c10d5e2c6a P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 10:49:49.941063 6787 raft_consensus.cc:482] T 501dceadcab94f87a23022c10d5e2c6a P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 10:49:49.941197 6787 raft_consensus.cc:3037] T 501dceadcab94f87a23022c10d5e2c6a P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:49:49.944120 6787 raft_consensus.cc:504] T 501dceadcab94f87a23022c10d5e2c6a P 3519d7d61a6f4a50909174f45b035d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:49.944876 6787 leader_election.cc:288] T 501dceadcab94f87a23022c10d5e2c6a P 3519d7d61a6f4a50909174f45b035d9b [CANDIDATE]: Term 1 election: Requested vote from peers ea7e70332cf64545889d381b59a7d558 (127.3.73.193:44797), dd8042b5b9ce4a61a5ca306878f9ab7a (127.3.73.194:41391)
I20260807 10:49:49.945431 6463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "501dceadcab94f87a23022c10d5e2c6a" candidate_uuid: "3519d7d61a6f4a50909174f45b035d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a"
I20260807 10:49:49.945473 6264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "501dceadcab94f87a23022c10d5e2c6a" candidate_uuid: "3519d7d61a6f4a50909174f45b035d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea7e70332cf64545889d381b59a7d558"
I20260807 10:49:49.945708 6463 raft_consensus.cc:3037] T 501dceadcab94f87a23022c10d5e2c6a P dd8042b5b9ce4a61a5ca306878f9ab7a [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:49:49.945713 6264 raft_consensus.cc:3037] T 501dceadcab94f87a23022c10d5e2c6a P ea7e70332cf64545889d381b59a7d558 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:49:49.948159 6463 raft_consensus.cc:2454] T 501dceadcab94f87a23022c10d5e2c6a P dd8042b5b9ce4a61a5ca306878f9ab7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3519d7d61a6f4a50909174f45b035d9b in term 1.
I20260807 10:49:49.948186 6264 raft_consensus.cc:2454] T 501dceadcab94f87a23022c10d5e2c6a P ea7e70332cf64545889d381b59a7d558 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3519d7d61a6f4a50909174f45b035d9b in term 1.
I20260807 10:49:49.948783 6567 leader_election.cc:302] T 501dceadcab94f87a23022c10d5e2c6a P 3519d7d61a6f4a50909174f45b035d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3519d7d61a6f4a50909174f45b035d9b, dd8042b5b9ce4a61a5ca306878f9ab7a, ea7e70332cf64545889d381b59a7d558; no voters:
I20260807 10:49:49.949126 6787 raft_consensus.cc:2781] T 501dceadcab94f87a23022c10d5e2c6a P 3519d7d61a6f4a50909174f45b035d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:49:49.949627 6787 raft_consensus.cc:686] T 501dceadcab94f87a23022c10d5e2c6a P 3519d7d61a6f4a50909174f45b035d9b [term 1 LEADER]: Becoming Leader. State: Replica: 3519d7d61a6f4a50909174f45b035d9b, State: Running, Role: LEADER
I20260807 10:49:49.950531 6787 consensus_queue.cc:237] T 501dceadcab94f87a23022c10d5e2c6a P 3519d7d61a6f4a50909174f45b035d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:49.956221 6827 tablet_replica.cc:439] T 501dceadcab94f87a23022c10d5e2c6a P 3519d7d61a6f4a50909174f45b035d9b: TxnStatusTablet state changed. Reason: New leader 3519d7d61a6f4a50909174f45b035d9b. Latest consensus state: current_term: 1 leader_uuid: "3519d7d61a6f4a50909174f45b035d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } } }
I20260807 10:49:49.957474 6827 tablet_replica.cc:442] T 501dceadcab94f87a23022c10d5e2c6a P 3519d7d61a6f4a50909174f45b035d9b: This TxnStatusTablet replica's current role is: LEADER
I20260807 10:49:49.962508 6829 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 10:49:49.963424 6829 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 10:49:49.968012 6008 catalog_manager.cc:5495] T 501dceadcab94f87a23022c10d5e2c6a P 3519d7d61a6f4a50909174f45b035d9b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3519d7d61a6f4a50909174f45b035d9b (127.3.73.195). New cstate: current_term: 1 leader_uuid: "3519d7d61a6f4a50909174f45b035d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } health_report { overall_health: HEALTHY } } }
I20260807 10:49:50.029290 6810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:50.012308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:39467 (local address 127.0.0.1:38252)
0807 10:49:50.013519 (+ 1211us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:50.013537 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:50.013794 (+ 257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:50.015042 (+ 1248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:50.015046 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:50.015084 (+ 38us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 10:49:50.015311 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:50.015321 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:50.019339 (+ 4018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:50.019344 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:50.021763 (+ 2419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:50.021792 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:50.024356 (+ 2564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:50.024361 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:50.024538 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:50.024605 (+ 67us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 10:49:50.026789 (+ 2184us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 10:49:50.026879 (+ 90us) client_negotiation.cc:769] Sending connection context
0807 10:49:50.027018 (+ 139us) client_negotiation.cc:240] Negotiation successful
0807 10:49:50.027353 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":867}
I20260807 10:49:50.029892 6772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:50.012353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38252 (local address 127.3.73.195:39467)
0807 10:49:50.013218 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:50.013246 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:50.013716 (+ 470us) server_negotiation.cc:407] Connection header received
0807 10:49:50.014511 (+ 795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:50.014539 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:50.014766 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:50.015031 (+ 265us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:49:50.015614 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:50.019097 (+ 3483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:50.022030 (+ 2933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:50.024143 (+ 2113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:50.024322 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:50.024851 (+ 529us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:49:50.026561 (+ 1710us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:49:50.026926 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:50.027163 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 10:49:50.027715 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240}
I20260807 10:49:50.058524 6463 raft_consensus.cc:1261] T 501dceadcab94f87a23022c10d5e2c6a P dd8042b5b9ce4a61a5ca306878f9ab7a [term 1 FOLLOWER]: Refusing update from remote peer 3519d7d61a6f4a50909174f45b035d9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:49:50.058553 6264 raft_consensus.cc:1261] T 501dceadcab94f87a23022c10d5e2c6a P ea7e70332cf64545889d381b59a7d558 [term 1 FOLLOWER]: Refusing update from remote peer 3519d7d61a6f4a50909174f45b035d9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:49:50.062216 6787 consensus_queue.cc:1035] T 501dceadcab94f87a23022c10d5e2c6a P 3519d7d61a6f4a50909174f45b035d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 10:49:50.065596 6787 consensus_queue.cc:1035] T 501dceadcab94f87a23022c10d5e2c6a P 3519d7d61a6f4a50909174f45b035d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260807 10:49:50.113721 6784 tablet_replica.cc:439] T 501dceadcab94f87a23022c10d5e2c6a P dd8042b5b9ce4a61a5ca306878f9ab7a: TxnStatusTablet state changed. Reason: New leader 3519d7d61a6f4a50909174f45b035d9b. Latest consensus state: current_term: 1 leader_uuid: "3519d7d61a6f4a50909174f45b035d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } } }
I20260807 10:49:50.114630 6784 tablet_replica.cc:442] T 501dceadcab94f87a23022c10d5e2c6a P dd8042b5b9ce4a61a5ca306878f9ab7a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 10:49:50.125433 6832 mvcc.cc:205] Tried to move back new op lower bound from 7315864740052004864 to 7315864739642368000. Current Snapshot: MvccSnapshot[applied={T|T < 7315864740052004864}]
I20260807 10:49:50.133396 6819 tablet_replica.cc:439] T 501dceadcab94f87a23022c10d5e2c6a P ea7e70332cf64545889d381b59a7d558: TxnStatusTablet state changed. Reason: New leader 3519d7d61a6f4a50909174f45b035d9b. Latest consensus state: current_term: 1 leader_uuid: "3519d7d61a6f4a50909174f45b035d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } } }
I20260807 10:49:50.133989 6787 tablet_replica.cc:439] T 501dceadcab94f87a23022c10d5e2c6a P 3519d7d61a6f4a50909174f45b035d9b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3519d7d61a6f4a50909174f45b035d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } } }
I20260807 10:49:50.135478 6787 tablet_replica.cc:442] T 501dceadcab94f87a23022c10d5e2c6a P 3519d7d61a6f4a50909174f45b035d9b: This TxnStatusTablet replica's current role is: LEADER
I20260807 10:49:50.139817 6819 tablet_replica.cc:442] T 501dceadcab94f87a23022c10d5e2c6a P ea7e70332cf64545889d381b59a7d558: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 10:49:50.142271 6784 tablet_replica.cc:439] T 501dceadcab94f87a23022c10d5e2c6a P dd8042b5b9ce4a61a5ca306878f9ab7a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3519d7d61a6f4a50909174f45b035d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } } }
I20260807 10:49:50.143214 6784 tablet_replica.cc:442] T 501dceadcab94f87a23022c10d5e2c6a P dd8042b5b9ce4a61a5ca306878f9ab7a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 10:49:50.148417 6787 tablet_replica.cc:439] T 501dceadcab94f87a23022c10d5e2c6a P 3519d7d61a6f4a50909174f45b035d9b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3519d7d61a6f4a50909174f45b035d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } } }
I20260807 10:49:50.152254 6787 tablet_replica.cc:442] T 501dceadcab94f87a23022c10d5e2c6a P 3519d7d61a6f4a50909174f45b035d9b: This TxnStatusTablet replica's current role is: LEADER
I20260807 10:49:50.174296 6819 tablet_replica.cc:439] T 501dceadcab94f87a23022c10d5e2c6a P ea7e70332cf64545889d381b59a7d558: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3519d7d61a6f4a50909174f45b035d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } } }
I20260807 10:49:50.174813 6819 tablet_replica.cc:442] T 501dceadcab94f87a23022c10d5e2c6a P ea7e70332cf64545889d381b59a7d558: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 10:49:50.177577 6784 consensus_queue.cc:1035] T 3a82b120b6e640a58a589c1c96abf3ae P dd8042b5b9ce4a61a5ca306878f9ab7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:49:50.208966 6849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:50.180471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53260 (local address 127.3.73.194:41391)
0807 10:49:50.184016 (+ 3545us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:50.184027 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:50.190179 (+ 6152us) server_negotiation.cc:407] Connection header received
0807 10:49:50.191207 (+ 1028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:50.191235 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:50.191616 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:50.192138 (+ 522us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:49:50.194113 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:50.196167 (+ 2054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:50.200078 (+ 3911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:50.201770 (+ 1692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:50.202453 (+ 683us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:50.204888 (+ 2435us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:49:50.206723 (+ 1835us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:49:50.207117 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:50.207884 (+ 767us) server_negotiation.cc:299] Negotiation successful
0807 10:49:50.208266 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3043,"thread_start_us":3476,"threads_started":1}
I20260807 10:49:50.239593 6855 consensus_queue.cc:1035] T 3a82b120b6e640a58a589c1c96abf3ae P dd8042b5b9ce4a61a5ca306878f9ab7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260807 10:49:50.242902 6353 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.73.195:39467, user_credentials={real_user=slave}} blocked reactor thread for 45326.9us
W20260807 10:49:50.258258 6358 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.3.73.193:44797, user_credentials={real_user=slave}} blocked reactor thread for 61389.9us
I20260807 10:49:50.314111 6772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:50.262438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:33735 (local address 127.3.73.195:39467)
0807 10:49:50.263485 (+ 1047us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:50.263538 (+ 53us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:50.280146 (+ 16608us) server_negotiation.cc:407] Connection header received
0807 10:49:50.282233 (+ 2087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:50.282241 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:50.282706 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:50.283224 (+ 518us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:49:50.287812 (+ 4588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:50.292577 (+ 4765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:50.302817 (+ 10240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:50.304702 (+ 1885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:50.305340 (+ 638us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:50.309769 (+ 4429us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:49:50.310816 (+ 1047us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:49:50.311494 (+ 678us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:50.312348 (+ 854us) server_negotiation.cc:299] Negotiation successful
0807 10:49:50.312643 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304}
I20260807 10:49:50.315253 6858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:50.260053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:39467 (local address 127.3.73.194:33735)
0807 10:49:50.279573 (+ 19520us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:50.279671 (+ 98us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:50.281268 (+ 1597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:50.284134 (+ 2866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:50.284167 (+ 33us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:50.284184 (+ 17us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 10:49:50.286845 (+ 2661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:50.286920 (+ 75us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:50.293629 (+ 6709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:50.293654 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:50.302488 (+ 8834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:50.302512 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:50.306531 (+ 4019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:50.306558 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:50.307901 (+ 1343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:50.308442 (+ 541us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 10:49:50.311838 (+ 3396us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 10:49:50.311904 (+ 66us) client_negotiation.cc:769] Sending connection context
0807 10:49:50.312210 (+ 306us) client_negotiation.cc:240] Negotiation successful
0807 10:49:50.312917 (+ 707us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18195,"thread_start_us":16344,"threads_started":1}
I20260807 10:49:50.482563 6864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:50.452342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:41391 (local address 127.3.73.195:45115)
0807 10:49:50.455694 (+ 3352us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:50.455726 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:50.456044 (+ 318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:50.457208 (+ 1164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:50.457224 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:50.457248 (+ 24us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 10:49:50.458355 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:50.458403 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:50.463995 (+ 5592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:50.464002 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:50.470144 (+ 6142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:50.470253 (+ 109us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:50.475728 (+ 5475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:50.475749 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:50.476448 (+ 699us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:50.476637 (+ 189us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 10:49:50.478676 (+ 2039us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 10:49:50.478726 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 10:49:50.478969 (+ 243us) client_negotiation.cc:240] Negotiation successful
0807 10:49:50.480030 (+ 1061us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2983,"thread_start_us":2710,"threads_started":1}
I20260807 10:49:50.484205 6849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:50.452369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:45115 (local address 127.3.73.194:41391)
0807 10:49:50.453261 (+ 892us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:50.453310 (+ 49us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:50.455866 (+ 2556us) server_negotiation.cc:407] Connection header received
0807 10:49:50.456652 (+ 786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:50.456679 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:50.456959 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:50.457185 (+ 226us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:49:50.459183 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:50.463689 (+ 4506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:50.471028 (+ 7339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:50.475012 (+ 3984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:50.475709 (+ 697us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:50.477295 (+ 1586us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:49:50.478062 (+ 767us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:49:50.478575 (+ 513us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:50.480580 (+ 2005us) server_negotiation.cc:299] Negotiation successful
0807 10:49:50.482353 (+ 1773us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164}
I20260807 10:49:50.693892 6013 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:49:50.695421 6013 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:49:50.718786 6013 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8ea443e2ab0a47f1b65df7aeac9255e8: Sending DeleteTablet for 3 replicas of tablet 3a82b120b6e640a58a589c1c96abf3ae
I20260807 10:49:50.723295 6435 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a82b120b6e640a58a589c1c96abf3ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:49:50 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:53254
I20260807 10:49:50.729539 6220 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a82b120b6e640a58a589c1c96abf3ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:49:50 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:42858
I20260807 10:49:50.737892 6640 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a82b120b6e640a58a589c1c96abf3ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:49:50 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:38242
I20260807 10:49:50.746114 6008 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46874:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 10:49:50.748631 6008 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 10:49:50.766983 6875 tablet_replica.cc:330] T 3a82b120b6e640a58a589c1c96abf3ae P ea7e70332cf64545889d381b59a7d558: stopping tablet replica
I20260807 10:49:50.768846 6875 raft_consensus.cc:2229] T 3a82b120b6e640a58a589c1c96abf3ae P ea7e70332cf64545889d381b59a7d558 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:49:50.772619 6874 tablet_replica.cc:330] T 3a82b120b6e640a58a589c1c96abf3ae P 3519d7d61a6f4a50909174f45b035d9b: stopping tablet replica
I20260807 10:49:50.776263 6875 raft_consensus.cc:2258] T 3a82b120b6e640a58a589c1c96abf3ae P ea7e70332cf64545889d381b59a7d558 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:49:50.777868 6872 tablet_replica.cc:330] T 3a82b120b6e640a58a589c1c96abf3ae P dd8042b5b9ce4a61a5ca306878f9ab7a: stopping tablet replica
I20260807 10:49:50.778519 6874 raft_consensus.cc:2229] T 3a82b120b6e640a58a589c1c96abf3ae P 3519d7d61a6f4a50909174f45b035d9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:49:50.779453 6872 raft_consensus.cc:2229] T 3a82b120b6e640a58a589c1c96abf3ae P dd8042b5b9ce4a61a5ca306878f9ab7a [term 1 LEADER]: Raft consensus shutting down.
I20260807 10:49:50.779758 6874 raft_consensus.cc:2258] T 3a82b120b6e640a58a589c1c96abf3ae P 3519d7d61a6f4a50909174f45b035d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:49:50.788038 6874 ts_tablet_manager.cc:1905] T 3a82b120b6e640a58a589c1c96abf3ae P 3519d7d61a6f4a50909174f45b035d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:49:50.793752 6435 tablet_service.cc:1467] Processing CreateTablet for tablet 75819c7f12524b8a95af7d18ec01db9c (DEFAULT_TABLE table=test-table [id=7d1391501c60457e910dc27f325ba690]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:49:50.795306 6435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75819c7f12524b8a95af7d18ec01db9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:49:50.796265 6220 tablet_service.cc:1467] Processing CreateTablet for tablet 75819c7f12524b8a95af7d18ec01db9c (DEFAULT_TABLE table=test-table [id=7d1391501c60457e910dc27f325ba690]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:49:50.798565 6220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75819c7f12524b8a95af7d18ec01db9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:49:50.794237 6640 tablet_service.cc:1467] Processing CreateTablet for tablet 75819c7f12524b8a95af7d18ec01db9c (DEFAULT_TABLE table=test-table [id=7d1391501c60457e910dc27f325ba690]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:49:50.807091 6875 ts_tablet_manager.cc:1905] T 3a82b120b6e640a58a589c1c96abf3ae P ea7e70332cf64545889d381b59a7d558: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:49:50.808055 6640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 75819c7f12524b8a95af7d18ec01db9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:49:50.843317 6872 raft_consensus.cc:2258] T 3a82b120b6e640a58a589c1c96abf3ae P dd8042b5b9ce4a61a5ca306878f9ab7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:49:50.848368 6874 ts_tablet_manager.cc:1918] T 3a82b120b6e640a58a589c1c96abf3ae P 3519d7d61a6f4a50909174f45b035d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 10:49:50.853390 6874 log.cc:1196] T 3a82b120b6e640a58a589c1c96abf3ae P 3519d7d61a6f4a50909174f45b035d9b: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-2/wal/wals/3a82b120b6e640a58a589c1c96abf3ae
I20260807 10:49:50.862249 6874 ts_tablet_manager.cc:1939] T 3a82b120b6e640a58a589c1c96abf3ae P 3519d7d61a6f4a50909174f45b035d9b: Deleting consensus metadata
I20260807 10:49:50.870565 6876 tablet_bootstrap.cc:492] T 75819c7f12524b8a95af7d18ec01db9c P dd8042b5b9ce4a61a5ca306878f9ab7a: Bootstrap starting.
I20260807 10:49:50.879376 6876 tablet_bootstrap.cc:654] T 75819c7f12524b8a95af7d18ec01db9c P dd8042b5b9ce4a61a5ca306878f9ab7a: Neither blocks nor log segments found. Creating new log.
I20260807 10:49:50.881191 6872 ts_tablet_manager.cc:1905] T 3a82b120b6e640a58a589c1c96abf3ae P dd8042b5b9ce4a61a5ca306878f9ab7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:49:50.884858 6875 ts_tablet_manager.cc:1918] T 3a82b120b6e640a58a589c1c96abf3ae P ea7e70332cf64545889d381b59a7d558: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 10:49:50.885591 6875 log.cc:1196] T 3a82b120b6e640a58a589c1c96abf3ae P ea7e70332cf64545889d381b59a7d558: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-0/wal/wals/3a82b120b6e640a58a589c1c96abf3ae
I20260807 10:49:50.889364 6875 ts_tablet_manager.cc:1939] T 3a82b120b6e640a58a589c1c96abf3ae P ea7e70332cf64545889d381b59a7d558: Deleting consensus metadata
I20260807 10:49:50.890122 5984 catalog_manager.cc:4841] TS 3519d7d61a6f4a50909174f45b035d9b (127.3.73.195:39467): tablet 3a82b120b6e640a58a589c1c96abf3ae (table test-table [id=d9d4749e46554e759549f0482a9bc377]) successfully deleted
I20260807 10:49:50.902640 5986 catalog_manager.cc:4841] TS ea7e70332cf64545889d381b59a7d558 (127.3.73.193:44797): tablet 3a82b120b6e640a58a589c1c96abf3ae (table test-table [id=d9d4749e46554e759549f0482a9bc377]) successfully deleted
I20260807 10:49:50.935372 6876 tablet_bootstrap.cc:492] T 75819c7f12524b8a95af7d18ec01db9c P dd8042b5b9ce4a61a5ca306878f9ab7a: No bootstrap required, opened a new log
I20260807 10:49:50.936272 6876 ts_tablet_manager.cc:1397] T 75819c7f12524b8a95af7d18ec01db9c P dd8042b5b9ce4a61a5ca306878f9ab7a: Time spent bootstrapping tablet: real 0.070s user 0.011s sys 0.016s
I20260807 10:49:50.941002 6876 raft_consensus.cc:348] T 75819c7f12524b8a95af7d18ec01db9c P dd8042b5b9ce4a61a5ca306878f9ab7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:50.941972 6876 raft_consensus.cc:374] T 75819c7f12524b8a95af7d18ec01db9c P dd8042b5b9ce4a61a5ca306878f9ab7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:49:50.942252 6876 raft_consensus.cc:729] T 75819c7f12524b8a95af7d18ec01db9c P dd8042b5b9ce4a61a5ca306878f9ab7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd8042b5b9ce4a61a5ca306878f9ab7a, State: Initialized, Role: FOLLOWER
I20260807 10:49:50.952137 6876 consensus_queue.cc:260] T 75819c7f12524b8a95af7d18ec01db9c P dd8042b5b9ce4a61a5ca306878f9ab7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:50.955211 6876 ts_tablet_manager.cc:1428] T 75819c7f12524b8a95af7d18ec01db9c P dd8042b5b9ce4a61a5ca306878f9ab7a: Time spent starting tablet: real 0.018s user 0.011s sys 0.000s
I20260807 10:49:51.000103 6880 tablet_bootstrap.cc:492] T 75819c7f12524b8a95af7d18ec01db9c P ea7e70332cf64545889d381b59a7d558: Bootstrap starting.
I20260807 10:49:51.002214 6881 tablet_bootstrap.cc:492] T 75819c7f12524b8a95af7d18ec01db9c P 3519d7d61a6f4a50909174f45b035d9b: Bootstrap starting.
I20260807 10:49:51.022989 6880 tablet_bootstrap.cc:654] T 75819c7f12524b8a95af7d18ec01db9c P ea7e70332cf64545889d381b59a7d558: Neither blocks nor log segments found. Creating new log.
I20260807 10:49:51.023490 6881 tablet_bootstrap.cc:654] T 75819c7f12524b8a95af7d18ec01db9c P 3519d7d61a6f4a50909174f45b035d9b: Neither blocks nor log segments found. Creating new log.
I20260807 10:49:51.039471 6881 tablet_bootstrap.cc:492] T 75819c7f12524b8a95af7d18ec01db9c P 3519d7d61a6f4a50909174f45b035d9b: No bootstrap required, opened a new log
I20260807 10:49:51.039906 6881 ts_tablet_manager.cc:1397] T 75819c7f12524b8a95af7d18ec01db9c P 3519d7d61a6f4a50909174f45b035d9b: Time spent bootstrapping tablet: real 0.038s user 0.008s sys 0.005s
I20260807 10:49:51.050412 6872 ts_tablet_manager.cc:1918] T 3a82b120b6e640a58a589c1c96abf3ae P dd8042b5b9ce4a61a5ca306878f9ab7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 10:49:51.051862 6872 log.cc:1196] T 3a82b120b6e640a58a589c1c96abf3ae P dd8042b5b9ce4a61a5ca306878f9ab7a: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-1/wal/wals/3a82b120b6e640a58a589c1c96abf3ae
I20260807 10:49:51.052878 6881 raft_consensus.cc:348] T 75819c7f12524b8a95af7d18ec01db9c P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:51.054091 6881 raft_consensus.cc:374] T 75819c7f12524b8a95af7d18ec01db9c P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:49:51.054418 6881 raft_consensus.cc:729] T 75819c7f12524b8a95af7d18ec01db9c P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3519d7d61a6f4a50909174f45b035d9b, State: Initialized, Role: FOLLOWER
I20260807 10:49:51.058511 6872 ts_tablet_manager.cc:1939] T 3a82b120b6e640a58a589c1c96abf3ae P dd8042b5b9ce4a61a5ca306878f9ab7a: Deleting consensus metadata
I20260807 10:49:51.056710 6881 consensus_queue.cc:260] T 75819c7f12524b8a95af7d18ec01db9c P 3519d7d61a6f4a50909174f45b035d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:51.069691 5984 catalog_manager.cc:4841] TS dd8042b5b9ce4a61a5ca306878f9ab7a (127.3.73.194:41391): tablet 3a82b120b6e640a58a589c1c96abf3ae (table test-table [id=d9d4749e46554e759549f0482a9bc377]) successfully deleted
I20260807 10:49:51.073028 6881 ts_tablet_manager.cc:1428] T 75819c7f12524b8a95af7d18ec01db9c P 3519d7d61a6f4a50909174f45b035d9b: Time spent starting tablet: real 0.032s user 0.002s sys 0.010s
I20260807 10:49:51.090308 6852 raft_consensus.cc:482] T 75819c7f12524b8a95af7d18ec01db9c P dd8042b5b9ce4a61a5ca306878f9ab7a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 10:49:51.091892 6852 raft_consensus.cc:504] T 75819c7f12524b8a95af7d18ec01db9c P dd8042b5b9ce4a61a5ca306878f9ab7a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:51.094535 6880 tablet_bootstrap.cc:492] T 75819c7f12524b8a95af7d18ec01db9c P ea7e70332cf64545889d381b59a7d558: No bootstrap required, opened a new log
I20260807 10:49:51.095556 6880 ts_tablet_manager.cc:1397] T 75819c7f12524b8a95af7d18ec01db9c P ea7e70332cf64545889d381b59a7d558: Time spent bootstrapping tablet: real 0.096s user 0.023s sys 0.010s
I20260807 10:49:51.096542 6787 raft_consensus.cc:482] T 75819c7f12524b8a95af7d18ec01db9c P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 10:49:51.097193 6787 raft_consensus.cc:504] T 75819c7f12524b8a95af7d18ec01db9c P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:51.098130 6264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75819c7f12524b8a95af7d18ec01db9c" candidate_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea7e70332cf64545889d381b59a7d558" is_pre_election: true
I20260807 10:49:51.098358 6787 leader_election.cc:288] T 75819c7f12524b8a95af7d18ec01db9c P 3519d7d61a6f4a50909174f45b035d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea7e70332cf64545889d381b59a7d558 (127.3.73.193:44797), dd8042b5b9ce4a61a5ca306878f9ab7a (127.3.73.194:41391)
I20260807 10:49:51.098503 6880 raft_consensus.cc:348] T 75819c7f12524b8a95af7d18ec01db9c P ea7e70332cf64545889d381b59a7d558 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:51.098912 6880 raft_consensus.cc:374] T 75819c7f12524b8a95af7d18ec01db9c P ea7e70332cf64545889d381b59a7d558 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:49:51.099479 6263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75819c7f12524b8a95af7d18ec01db9c" candidate_uuid: "3519d7d61a6f4a50909174f45b035d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea7e70332cf64545889d381b59a7d558" is_pre_election: true
I20260807 10:49:51.100461 6670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75819c7f12524b8a95af7d18ec01db9c" candidate_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3519d7d61a6f4a50909174f45b035d9b" is_pre_election: true
I20260807 10:49:51.101517 6670 raft_consensus.cc:2454] T 75819c7f12524b8a95af7d18ec01db9c P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd8042b5b9ce4a61a5ca306878f9ab7a in term 0.
I20260807 10:49:51.102732 6462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75819c7f12524b8a95af7d18ec01db9c" candidate_uuid: "3519d7d61a6f4a50909174f45b035d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" is_pre_election: true
I20260807 10:49:51.103622 6462 raft_consensus.cc:2454] T 75819c7f12524b8a95af7d18ec01db9c P dd8042b5b9ce4a61a5ca306878f9ab7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3519d7d61a6f4a50909174f45b035d9b in term 0.
I20260807 10:49:51.106115 6567 leader_election.cc:302] T 75819c7f12524b8a95af7d18ec01db9c P 3519d7d61a6f4a50909174f45b035d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3519d7d61a6f4a50909174f45b035d9b, dd8042b5b9ce4a61a5ca306878f9ab7a; no voters: ea7e70332cf64545889d381b59a7d558
I20260807 10:49:51.108446 6787 raft_consensus.cc:2781] T 75819c7f12524b8a95af7d18ec01db9c P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 10:49:51.109207 6787 raft_consensus.cc:482] T 75819c7f12524b8a95af7d18ec01db9c P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 10:49:51.109576 6787 raft_consensus.cc:3037] T 75819c7f12524b8a95af7d18ec01db9c P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:49:51.110400 6353 leader_election.cc:302] T 75819c7f12524b8a95af7d18ec01db9c P dd8042b5b9ce4a61a5ca306878f9ab7a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3519d7d61a6f4a50909174f45b035d9b, dd8042b5b9ce4a61a5ca306878f9ab7a; no voters:
I20260807 10:49:51.114779 6852 leader_election.cc:288] T 75819c7f12524b8a95af7d18ec01db9c P dd8042b5b9ce4a61a5ca306878f9ab7a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea7e70332cf64545889d381b59a7d558 (127.3.73.193:44797), 3519d7d61a6f4a50909174f45b035d9b (127.3.73.195:39467)
I20260807 10:49:51.115852 6852 raft_consensus.cc:2781] T 75819c7f12524b8a95af7d18ec01db9c P dd8042b5b9ce4a61a5ca306878f9ab7a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 10:49:51.116683 6852 raft_consensus.cc:482] T 75819c7f12524b8a95af7d18ec01db9c P dd8042b5b9ce4a61a5ca306878f9ab7a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 10:49:51.120579 6852 raft_consensus.cc:3037] T 75819c7f12524b8a95af7d18ec01db9c P dd8042b5b9ce4a61a5ca306878f9ab7a [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:49:51.099102 6880 raft_consensus.cc:729] T 75819c7f12524b8a95af7d18ec01db9c P ea7e70332cf64545889d381b59a7d558 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea7e70332cf64545889d381b59a7d558, State: Initialized, Role: FOLLOWER
I20260807 10:49:51.126590 6787 raft_consensus.cc:504] T 75819c7f12524b8a95af7d18ec01db9c P 3519d7d61a6f4a50909174f45b035d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:51.126457 6880 consensus_queue.cc:260] T 75819c7f12524b8a95af7d18ec01db9c P ea7e70332cf64545889d381b59a7d558 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:51.129201 6264 raft_consensus.cc:2454] T 75819c7f12524b8a95af7d18ec01db9c P ea7e70332cf64545889d381b59a7d558 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd8042b5b9ce4a61a5ca306878f9ab7a in term 0.
I20260807 10:49:51.138120 6264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75819c7f12524b8a95af7d18ec01db9c" candidate_uuid: "3519d7d61a6f4a50909174f45b035d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea7e70332cf64545889d381b59a7d558"
I20260807 10:49:51.139181 6264 raft_consensus.cc:3037] T 75819c7f12524b8a95af7d18ec01db9c P ea7e70332cf64545889d381b59a7d558 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:49:51.147183 6787 leader_election.cc:288] T 75819c7f12524b8a95af7d18ec01db9c P 3519d7d61a6f4a50909174f45b035d9b [CANDIDATE]: Term 1 election: Requested vote from peers ea7e70332cf64545889d381b59a7d558 (127.3.73.193:44797), dd8042b5b9ce4a61a5ca306878f9ab7a (127.3.73.194:41391)
I20260807 10:49:51.154064 6462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75819c7f12524b8a95af7d18ec01db9c" candidate_uuid: "3519d7d61a6f4a50909174f45b035d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a"
I20260807 10:49:51.159672 6264 raft_consensus.cc:2454] T 75819c7f12524b8a95af7d18ec01db9c P ea7e70332cf64545889d381b59a7d558 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3519d7d61a6f4a50909174f45b035d9b in term 1.
I20260807 10:49:51.162070 6570 leader_election.cc:302] T 75819c7f12524b8a95af7d18ec01db9c P 3519d7d61a6f4a50909174f45b035d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3519d7d61a6f4a50909174f45b035d9b, ea7e70332cf64545889d381b59a7d558; no voters:
I20260807 10:49:51.163733 6787 raft_consensus.cc:2781] T 75819c7f12524b8a95af7d18ec01db9c P 3519d7d61a6f4a50909174f45b035d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:49:51.168429 6880 ts_tablet_manager.cc:1428] T 75819c7f12524b8a95af7d18ec01db9c P ea7e70332cf64545889d381b59a7d558: Time spent starting tablet: real 0.072s user 0.003s sys 0.009s
I20260807 10:49:51.178520 6787 raft_consensus.cc:686] T 75819c7f12524b8a95af7d18ec01db9c P 3519d7d61a6f4a50909174f45b035d9b [term 1 LEADER]: Becoming Leader. State: Replica: 3519d7d61a6f4a50909174f45b035d9b, State: Running, Role: LEADER
I20260807 10:49:51.179482 6787 consensus_queue.cc:237] T 75819c7f12524b8a95af7d18ec01db9c P 3519d7d61a6f4a50909174f45b035d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:51.173884 6852 raft_consensus.cc:504] T 75819c7f12524b8a95af7d18ec01db9c P dd8042b5b9ce4a61a5ca306878f9ab7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:51.185981 6462 raft_consensus.cc:2379] T 75819c7f12524b8a95af7d18ec01db9c P dd8042b5b9ce4a61a5ca306878f9ab7a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3519d7d61a6f4a50909174f45b035d9b in current term 1: Already voted for candidate dd8042b5b9ce4a61a5ca306878f9ab7a in this term.
I20260807 10:49:51.196509 6013 catalog_manager.cc:5495] T 75819c7f12524b8a95af7d18ec01db9c P 3519d7d61a6f4a50909174f45b035d9b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3519d7d61a6f4a50909174f45b035d9b (127.3.73.195). New cstate: current_term: 1 leader_uuid: "3519d7d61a6f4a50909174f45b035d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } health_report { overall_health: HEALTHY } } }
I20260807 10:49:51.200486 6264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75819c7f12524b8a95af7d18ec01db9c" candidate_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea7e70332cf64545889d381b59a7d558"
I20260807 10:49:51.201812 6264 raft_consensus.cc:2379] T 75819c7f12524b8a95af7d18ec01db9c P ea7e70332cf64545889d381b59a7d558 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate dd8042b5b9ce4a61a5ca306878f9ab7a in current term 1: Already voted for candidate 3519d7d61a6f4a50909174f45b035d9b in this term.
I20260807 10:49:51.201802 6670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "75819c7f12524b8a95af7d18ec01db9c" candidate_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3519d7d61a6f4a50909174f45b035d9b"
I20260807 10:49:51.205956 6353 leader_election.cc:302] T 75819c7f12524b8a95af7d18ec01db9c P dd8042b5b9ce4a61a5ca306878f9ab7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd8042b5b9ce4a61a5ca306878f9ab7a; no voters: 3519d7d61a6f4a50909174f45b035d9b, ea7e70332cf64545889d381b59a7d558
I20260807 10:49:51.207016 6852 leader_election.cc:288] T 75819c7f12524b8a95af7d18ec01db9c P dd8042b5b9ce4a61a5ca306878f9ab7a [CANDIDATE]: Term 1 election: Requested vote from peers ea7e70332cf64545889d381b59a7d558 (127.3.73.193:44797), 3519d7d61a6f4a50909174f45b035d9b (127.3.73.195:39467)
I20260807 10:49:51.207055 6885 raft_consensus.cc:2726] T 75819c7f12524b8a95af7d18ec01db9c P dd8042b5b9ce4a61a5ca306878f9ab7a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260807 10:49:51.416237 6908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:51.381282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35986 (local address 127.3.73.195:39467)
0807 10:49:51.391297 (+ 10015us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:51.391319 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:51.391393 (+ 74us) server_negotiation.cc:407] Connection header received
0807 10:49:51.391570 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:51.391583 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:51.391821 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:51.392074 (+ 253us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:49:51.394699 (+ 2625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:51.398454 (+ 3755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:51.406894 (+ 8440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:51.409969 (+ 3075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:51.412256 (+ 2287us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:51.413247 (+ 991us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:49:51.414010 (+ 763us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:49:51.414220 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:51.414829 (+ 609us) server_negotiation.cc:299] Negotiation successful
0807 10:49:51.415103 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9435,"thread_start_us":9022,"threads_started":1}
I20260807 10:49:51.492844 6908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:51.456000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:35715 (local address 127.3.73.195:39467)
0807 10:49:51.456557 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:51.456600 (+ 43us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:51.458081 (+ 1481us) server_negotiation.cc:407] Connection header received
0807 10:49:51.459090 (+ 1009us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:51.459144 (+ 54us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:51.459516 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:51.459807 (+ 291us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:49:51.463786 (+ 3979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:51.469954 (+ 6168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:51.479696 (+ 9742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:51.484582 (+ 4886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:51.485508 (+ 926us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:51.488432 (+ 2924us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:49:51.488810 (+ 378us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:49:51.489172 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:51.489773 (+ 601us) server_negotiation.cc:299] Negotiation successful
0807 10:49:51.490504 (+ 731us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260807 10:49:51.497102 6911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:51.455821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:39467 (local address 127.3.73.195:35715)
0807 10:49:51.457796 (+ 1975us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:51.457821 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:51.458422 (+ 601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:51.461106 (+ 2684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:51.461140 (+ 34us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:51.461214 (+ 74us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 10:49:51.462621 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:51.462685 (+ 64us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:51.471586 (+ 8901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:51.471596 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:51.478932 (+ 7336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:51.479032 (+ 100us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:51.486915 (+ 7883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:51.486934 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:51.487824 (+ 890us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:51.487939 (+ 115us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 10:49:51.489403 (+ 1464us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 10:49:51.489413 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 10:49:51.494377 (+ 4964us) client_negotiation.cc:240] Negotiation successful
0807 10:49:51.495443 (+ 1066us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1743,"thread_start_us":1374,"threads_started":1}
I20260807 10:49:51.551983 6463 raft_consensus.cc:1261] T 75819c7f12524b8a95af7d18ec01db9c P dd8042b5b9ce4a61a5ca306878f9ab7a [term 1 FOLLOWER]: Refusing update from remote peer 3519d7d61a6f4a50909174f45b035d9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:49:51.558059 6264 raft_consensus.cc:1261] T 75819c7f12524b8a95af7d18ec01db9c P ea7e70332cf64545889d381b59a7d558 [term 1 FOLLOWER]: Refusing update from remote peer 3519d7d61a6f4a50909174f45b035d9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:49:51.562753 6787 consensus_queue.cc:1035] T 75819c7f12524b8a95af7d18ec01db9c P 3519d7d61a6f4a50909174f45b035d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 10:49:51.568409 6887 consensus_queue.cc:1035] T 75819c7f12524b8a95af7d18ec01db9c P 3519d7d61a6f4a50909174f45b035d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260807 10:49:51.647476 6898 mvcc.cc:205] Tried to move back new op lower bound from 7315864746174283776 to 7315864744679882752. Current Snapshot: MvccSnapshot[applied={T|T < 7315864746174283776}]
W20260807 10:49:51.696264 6890 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 10:49:53.325640 6013 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:49:53.326083 6013 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:49:53.333323 6013 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8ea443e2ab0a47f1b65df7aeac9255e8: Sending DeleteTablet for 3 replicas of tablet 75819c7f12524b8a95af7d18ec01db9c
I20260807 10:49:53.334606 6220 tablet_service.cc:1514] Processing DeleteTablet for tablet 75819c7f12524b8a95af7d18ec01db9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:49:53 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:42858
I20260807 10:49:53.334658 6435 tablet_service.cc:1514] Processing DeleteTablet for tablet 75819c7f12524b8a95af7d18ec01db9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:49:53 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:53254
I20260807 10:49:53.334858 6640 tablet_service.cc:1514] Processing DeleteTablet for tablet 75819c7f12524b8a95af7d18ec01db9c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:49:53 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:38242
I20260807 10:49:53.346069 6947 tablet_replica.cc:330] T 75819c7f12524b8a95af7d18ec01db9c P ea7e70332cf64545889d381b59a7d558: stopping tablet replica
I20260807 10:49:53.346736 6947 raft_consensus.cc:2229] T 75819c7f12524b8a95af7d18ec01db9c P ea7e70332cf64545889d381b59a7d558 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:49:53.347399 6947 raft_consensus.cc:2258] T 75819c7f12524b8a95af7d18ec01db9c P ea7e70332cf64545889d381b59a7d558 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:49:53.349581 6947 ts_tablet_manager.cc:1905] T 75819c7f12524b8a95af7d18ec01db9c P ea7e70332cf64545889d381b59a7d558: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:49:53.357856 6948 tablet_replica.cc:330] T 75819c7f12524b8a95af7d18ec01db9c P 3519d7d61a6f4a50909174f45b035d9b: stopping tablet replica
I20260807 10:49:53.358304 6946 tablet_replica.cc:330] T 75819c7f12524b8a95af7d18ec01db9c P dd8042b5b9ce4a61a5ca306878f9ab7a: stopping tablet replica
I20260807 10:49:53.358351 6948 raft_consensus.cc:2229] T 75819c7f12524b8a95af7d18ec01db9c P 3519d7d61a6f4a50909174f45b035d9b [term 1 LEADER]: Raft consensus shutting down.
I20260807 10:49:53.358579 6946 raft_consensus.cc:2229] T 75819c7f12524b8a95af7d18ec01db9c P dd8042b5b9ce4a61a5ca306878f9ab7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:49:53.358948 6946 raft_consensus.cc:2258] T 75819c7f12524b8a95af7d18ec01db9c P dd8042b5b9ce4a61a5ca306878f9ab7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:49:53.359445 6948 raft_consensus.cc:2258] T 75819c7f12524b8a95af7d18ec01db9c P 3519d7d61a6f4a50909174f45b035d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:49:53.360599 6946 ts_tablet_manager.cc:1905] T 75819c7f12524b8a95af7d18ec01db9c P dd8042b5b9ce4a61a5ca306878f9ab7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:49:53.362450 6948 ts_tablet_manager.cc:1905] T 75819c7f12524b8a95af7d18ec01db9c P 3519d7d61a6f4a50909174f45b035d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:49:53.390568 6948 ts_tablet_manager.cc:1918] T 75819c7f12524b8a95af7d18ec01db9c P 3519d7d61a6f4a50909174f45b035d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 10:49:53.390679 6946 ts_tablet_manager.cc:1918] T 75819c7f12524b8a95af7d18ec01db9c P dd8042b5b9ce4a61a5ca306878f9ab7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 10:49:53.390892 6948 log.cc:1196] T 75819c7f12524b8a95af7d18ec01db9c P 3519d7d61a6f4a50909174f45b035d9b: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-2/wal/wals/75819c7f12524b8a95af7d18ec01db9c
I20260807 10:49:53.391173 6946 log.cc:1196] T 75819c7f12524b8a95af7d18ec01db9c P dd8042b5b9ce4a61a5ca306878f9ab7a: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-1/wal/wals/75819c7f12524b8a95af7d18ec01db9c
I20260807 10:49:53.391252 6947 ts_tablet_manager.cc:1918] T 75819c7f12524b8a95af7d18ec01db9c P ea7e70332cf64545889d381b59a7d558: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 10:49:53.391511 6947 log.cc:1196] T 75819c7f12524b8a95af7d18ec01db9c P ea7e70332cf64545889d381b59a7d558: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-0/wal/wals/75819c7f12524b8a95af7d18ec01db9c
I20260807 10:49:53.392927 6946 ts_tablet_manager.cc:1939] T 75819c7f12524b8a95af7d18ec01db9c P dd8042b5b9ce4a61a5ca306878f9ab7a: Deleting consensus metadata
I20260807 10:49:53.393033 6948 ts_tablet_manager.cc:1939] T 75819c7f12524b8a95af7d18ec01db9c P 3519d7d61a6f4a50909174f45b035d9b: Deleting consensus metadata
I20260807 10:49:53.393043 6947 ts_tablet_manager.cc:1939] T 75819c7f12524b8a95af7d18ec01db9c P ea7e70332cf64545889d381b59a7d558: Deleting consensus metadata
I20260807 10:49:53.394541 5984 catalog_manager.cc:4841] TS dd8042b5b9ce4a61a5ca306878f9ab7a (127.3.73.194:41391): tablet 75819c7f12524b8a95af7d18ec01db9c (table test-table [id=7d1391501c60457e910dc27f325ba690]) successfully deleted
I20260807 10:49:53.394688 5986 catalog_manager.cc:4841] TS ea7e70332cf64545889d381b59a7d558 (127.3.73.193:44797): tablet 75819c7f12524b8a95af7d18ec01db9c (table test-table [id=7d1391501c60457e910dc27f325ba690]) successfully deleted
I20260807 10:49:53.394918 5984 catalog_manager.cc:4841] TS 3519d7d61a6f4a50909174f45b035d9b (127.3.73.195:39467): tablet 75819c7f12524b8a95af7d18ec01db9c (table test-table [id=7d1391501c60457e910dc27f325ba690]) successfully deleted
Aug 07 10:49:53 dist-test-slave-x5fp krb5kdc[5938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099793, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 10:49:53 dist-test-slave-x5fp krb5kdc[5938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099793, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:49:53.432122 6958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:53.406959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37418 (local address 127.3.73.254:41773)
0807 10:49:53.407741 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:53.407750 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:53.407782 (+ 32us) server_negotiation.cc:407] Connection header received
0807 10:49:53.407879 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:53.407883 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:53.407981 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:53.408106 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:53.409599 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:53.410517 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:53.411919 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:53.412406 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:53.412540 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:53.424867 (+ 12327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:53.424878 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:53.424885 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:53.424936 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:53.428772 (+ 3836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:53.429625 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:53.429633 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:53.429637 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:53.429821 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:53.430273 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:53.430286 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:53.430288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:53.430660 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:53.431068 (+ 408us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:53.431382 (+ 314us) server_negotiation.cc:299] Negotiation successful
0807 10:49:53.431686 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":459,"threads_started":1}
I20260807 10:49:53.436972 6013 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37418:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 10:49:53.437816 6013 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 10:49:53.446040 6220 tablet_service.cc:1467] Processing CreateTablet for tablet cc93601c0e0542aaa489a6133205108e (DEFAULT_TABLE table=test-table [id=d5f4212c6db84f6ca2f4c082474f303b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:49:53.446317 6435 tablet_service.cc:1467] Processing CreateTablet for tablet cc93601c0e0542aaa489a6133205108e (DEFAULT_TABLE table=test-table [id=d5f4212c6db84f6ca2f4c082474f303b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:49:53.446338 6640 tablet_service.cc:1467] Processing CreateTablet for tablet cc93601c0e0542aaa489a6133205108e (DEFAULT_TABLE table=test-table [id=d5f4212c6db84f6ca2f4c082474f303b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:49:53.446677 6220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc93601c0e0542aaa489a6133205108e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:49:53.446784 6435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc93601c0e0542aaa489a6133205108e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:49:53.446869 6640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc93601c0e0542aaa489a6133205108e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:49:53.456458 6959 tablet_bootstrap.cc:492] T cc93601c0e0542aaa489a6133205108e P ea7e70332cf64545889d381b59a7d558: Bootstrap starting.
I20260807 10:49:53.456842 6961 tablet_bootstrap.cc:492] T cc93601c0e0542aaa489a6133205108e P dd8042b5b9ce4a61a5ca306878f9ab7a: Bootstrap starting.
I20260807 10:49:53.455366 6960 tablet_bootstrap.cc:492] T cc93601c0e0542aaa489a6133205108e P 3519d7d61a6f4a50909174f45b035d9b: Bootstrap starting.
I20260807 10:49:53.459453 6959 tablet_bootstrap.cc:654] T cc93601c0e0542aaa489a6133205108e P ea7e70332cf64545889d381b59a7d558: Neither blocks nor log segments found. Creating new log.
I20260807 10:49:53.460292 6961 tablet_bootstrap.cc:654] T cc93601c0e0542aaa489a6133205108e P dd8042b5b9ce4a61a5ca306878f9ab7a: Neither blocks nor log segments found. Creating new log.
I20260807 10:49:53.460656 6960 tablet_bootstrap.cc:654] T cc93601c0e0542aaa489a6133205108e P 3519d7d61a6f4a50909174f45b035d9b: Neither blocks nor log segments found. Creating new log.
I20260807 10:49:53.464687 6959 tablet_bootstrap.cc:492] T cc93601c0e0542aaa489a6133205108e P ea7e70332cf64545889d381b59a7d558: No bootstrap required, opened a new log
I20260807 10:49:53.464963 6959 ts_tablet_manager.cc:1397] T cc93601c0e0542aaa489a6133205108e P ea7e70332cf64545889d381b59a7d558: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.005s
I20260807 10:49:53.465958 6961 tablet_bootstrap.cc:492] T cc93601c0e0542aaa489a6133205108e P dd8042b5b9ce4a61a5ca306878f9ab7a: No bootstrap required, opened a new log
I20260807 10:49:53.466199 6961 ts_tablet_manager.cc:1397] T cc93601c0e0542aaa489a6133205108e P dd8042b5b9ce4a61a5ca306878f9ab7a: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260807 10:49:53.466432 6960 tablet_bootstrap.cc:492] T cc93601c0e0542aaa489a6133205108e P 3519d7d61a6f4a50909174f45b035d9b: No bootstrap required, opened a new log
I20260807 10:49:53.466710 6960 ts_tablet_manager.cc:1397] T cc93601c0e0542aaa489a6133205108e P 3519d7d61a6f4a50909174f45b035d9b: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260807 10:49:53.466564 6959 raft_consensus.cc:348] T cc93601c0e0542aaa489a6133205108e P ea7e70332cf64545889d381b59a7d558 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:53.467043 6959 raft_consensus.cc:374] T cc93601c0e0542aaa489a6133205108e P ea7e70332cf64545889d381b59a7d558 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:49:53.467324 6959 raft_consensus.cc:729] T cc93601c0e0542aaa489a6133205108e P ea7e70332cf64545889d381b59a7d558 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea7e70332cf64545889d381b59a7d558, State: Initialized, Role: FOLLOWER
I20260807 10:49:53.467713 6959 consensus_queue.cc:260] T cc93601c0e0542aaa489a6133205108e P ea7e70332cf64545889d381b59a7d558 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:53.467702 6961 raft_consensus.cc:348] T cc93601c0e0542aaa489a6133205108e P dd8042b5b9ce4a61a5ca306878f9ab7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:53.468130 6961 raft_consensus.cc:374] T cc93601c0e0542aaa489a6133205108e P dd8042b5b9ce4a61a5ca306878f9ab7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:49:53.468248 6961 raft_consensus.cc:729] T cc93601c0e0542aaa489a6133205108e P dd8042b5b9ce4a61a5ca306878f9ab7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd8042b5b9ce4a61a5ca306878f9ab7a, State: Initialized, Role: FOLLOWER
I20260807 10:49:53.468119 6960 raft_consensus.cc:348] T cc93601c0e0542aaa489a6133205108e P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:53.468585 6960 raft_consensus.cc:374] T cc93601c0e0542aaa489a6133205108e P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:49:53.468768 6960 raft_consensus.cc:729] T cc93601c0e0542aaa489a6133205108e P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3519d7d61a6f4a50909174f45b035d9b, State: Initialized, Role: FOLLOWER
I20260807 10:49:53.468688 6961 consensus_queue.cc:260] T cc93601c0e0542aaa489a6133205108e P dd8042b5b9ce4a61a5ca306878f9ab7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:53.469354 6959 ts_tablet_manager.cc:1428] T cc93601c0e0542aaa489a6133205108e P ea7e70332cf64545889d381b59a7d558: Time spent starting tablet: real 0.004s user 0.001s sys 0.003s
I20260807 10:49:53.469110 6960 consensus_queue.cc:260] T cc93601c0e0542aaa489a6133205108e P 3519d7d61a6f4a50909174f45b035d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:53.470178 6960 ts_tablet_manager.cc:1428] T cc93601c0e0542aaa489a6133205108e P 3519d7d61a6f4a50909174f45b035d9b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 10:49:53.471139 6961 ts_tablet_manager.cc:1428] T cc93601c0e0542aaa489a6133205108e P dd8042b5b9ce4a61a5ca306878f9ab7a: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260807 10:49:53.684677 6969 raft_consensus.cc:482] T cc93601c0e0542aaa489a6133205108e P dd8042b5b9ce4a61a5ca306878f9ab7a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 10:49:53.685113 6969 raft_consensus.cc:504] T cc93601c0e0542aaa489a6133205108e P dd8042b5b9ce4a61a5ca306878f9ab7a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:53.687243 6969 leader_election.cc:288] T cc93601c0e0542aaa489a6133205108e P dd8042b5b9ce4a61a5ca306878f9ab7a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea7e70332cf64545889d381b59a7d558 (127.3.73.193:44797), 3519d7d61a6f4a50909174f45b035d9b (127.3.73.195:39467)
I20260807 10:49:53.687986 6264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc93601c0e0542aaa489a6133205108e" candidate_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea7e70332cf64545889d381b59a7d558" is_pre_election: true
I20260807 10:49:53.688112 6670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc93601c0e0542aaa489a6133205108e" candidate_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3519d7d61a6f4a50909174f45b035d9b" is_pre_election: true
I20260807 10:49:53.688534 6670 raft_consensus.cc:2454] T cc93601c0e0542aaa489a6133205108e P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd8042b5b9ce4a61a5ca306878f9ab7a in term 0.
I20260807 10:49:53.688522 6264 raft_consensus.cc:2454] T cc93601c0e0542aaa489a6133205108e P ea7e70332cf64545889d381b59a7d558 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd8042b5b9ce4a61a5ca306878f9ab7a in term 0.
I20260807 10:49:53.689131 6358 leader_election.cc:302] T cc93601c0e0542aaa489a6133205108e P dd8042b5b9ce4a61a5ca306878f9ab7a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dd8042b5b9ce4a61a5ca306878f9ab7a, ea7e70332cf64545889d381b59a7d558; no voters:
I20260807 10:49:53.689555 6969 raft_consensus.cc:2781] T cc93601c0e0542aaa489a6133205108e P dd8042b5b9ce4a61a5ca306878f9ab7a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 10:49:53.689844 6969 raft_consensus.cc:482] T cc93601c0e0542aaa489a6133205108e P dd8042b5b9ce4a61a5ca306878f9ab7a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 10:49:53.690105 6969 raft_consensus.cc:3037] T cc93601c0e0542aaa489a6133205108e P dd8042b5b9ce4a61a5ca306878f9ab7a [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:49:53.693813 6969 raft_consensus.cc:504] T cc93601c0e0542aaa489a6133205108e P dd8042b5b9ce4a61a5ca306878f9ab7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:53.694679 6969 leader_election.cc:288] T cc93601c0e0542aaa489a6133205108e P dd8042b5b9ce4a61a5ca306878f9ab7a [CANDIDATE]: Term 1 election: Requested vote from peers ea7e70332cf64545889d381b59a7d558 (127.3.73.193:44797), 3519d7d61a6f4a50909174f45b035d9b (127.3.73.195:39467)
I20260807 10:49:53.695362 6264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc93601c0e0542aaa489a6133205108e" candidate_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea7e70332cf64545889d381b59a7d558"
I20260807 10:49:53.695429 6670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc93601c0e0542aaa489a6133205108e" candidate_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3519d7d61a6f4a50909174f45b035d9b"
I20260807 10:49:53.695747 6670 raft_consensus.cc:3037] T cc93601c0e0542aaa489a6133205108e P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:49:53.695740 6264 raft_consensus.cc:3037] T cc93601c0e0542aaa489a6133205108e P ea7e70332cf64545889d381b59a7d558 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:49:53.698845 6670 raft_consensus.cc:2454] T cc93601c0e0542aaa489a6133205108e P 3519d7d61a6f4a50909174f45b035d9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd8042b5b9ce4a61a5ca306878f9ab7a in term 1.
I20260807 10:49:53.698941 6264 raft_consensus.cc:2454] T cc93601c0e0542aaa489a6133205108e P ea7e70332cf64545889d381b59a7d558 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd8042b5b9ce4a61a5ca306878f9ab7a in term 1.
I20260807 10:49:53.699482 6353 leader_election.cc:302] T cc93601c0e0542aaa489a6133205108e P dd8042b5b9ce4a61a5ca306878f9ab7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3519d7d61a6f4a50909174f45b035d9b, dd8042b5b9ce4a61a5ca306878f9ab7a; no voters:
I20260807 10:49:53.699924 6969 raft_consensus.cc:2781] T cc93601c0e0542aaa489a6133205108e P dd8042b5b9ce4a61a5ca306878f9ab7a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:49:53.700971 6969 raft_consensus.cc:686] T cc93601c0e0542aaa489a6133205108e P dd8042b5b9ce4a61a5ca306878f9ab7a [term 1 LEADER]: Becoming Leader. State: Replica: dd8042b5b9ce4a61a5ca306878f9ab7a, State: Running, Role: LEADER
I20260807 10:49:53.701398 6969 consensus_queue.cc:237] T cc93601c0e0542aaa489a6133205108e P dd8042b5b9ce4a61a5ca306878f9ab7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:53.705210 6008 catalog_manager.cc:5495] T cc93601c0e0542aaa489a6133205108e P dd8042b5b9ce4a61a5ca306878f9ab7a reported cstate change: term changed from 0 to 1, leader changed from <none> to dd8042b5b9ce4a61a5ca306878f9ab7a (127.3.73.194). New cstate: current_term: 1 leader_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } health_report { overall_health: UNKNOWN } } }
I20260807 10:49:53.780807 6976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:53.769073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37650 (local address 127.3.73.194:41391)
0807 10:49:53.770875 (+ 1802us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:53.770883 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:53.770955 (+ 72us) server_negotiation.cc:407] Connection header received
0807 10:49:53.771115 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:53.771121 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:53.771265 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:53.771459 (+ 194us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:49:53.772262 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:53.774430 (+ 2168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:53.776691 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:53.777649 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:53.778775 (+ 1126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:53.779017 (+ 242us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:49:53.779327 (+ 310us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:49:53.779988 (+ 661us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:53.780047 (+ 59us) server_negotiation.cc:299] Negotiation successful
0807 10:49:53.780122 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1465,"thread_start_us":1091,"threads_started":1}
I20260807 10:49:53.794596 6264 raft_consensus.cc:1261] T cc93601c0e0542aaa489a6133205108e P ea7e70332cf64545889d381b59a7d558 [term 1 FOLLOWER]: Refusing update from remote peer dd8042b5b9ce4a61a5ca306878f9ab7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:49:53.794595 6670 raft_consensus.cc:1261] T cc93601c0e0542aaa489a6133205108e P 3519d7d61a6f4a50909174f45b035d9b [term 1 FOLLOWER]: Refusing update from remote peer dd8042b5b9ce4a61a5ca306878f9ab7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:49:53.795591 6971 consensus_queue.cc:1035] T cc93601c0e0542aaa489a6133205108e P dd8042b5b9ce4a61a5ca306878f9ab7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 10:49:53.796191 6969 consensus_queue.cc:1035] T cc93601c0e0542aaa489a6133205108e P dd8042b5b9ce4a61a5ca306878f9ab7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:49:53.801889 6926 mvcc.cc:205] Tried to move back new op lower bound from 7315864755375263744 to 7315864755003637760. Current Snapshot: MvccSnapshot[applied={T|T < 7315864755375263744}]
I20260807 10:49:53.870430 6010 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:49:53.870745 6010 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:49:53.874681 6010 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8ea443e2ab0a47f1b65df7aeac9255e8: Sending DeleteTablet for 3 replicas of tablet cc93601c0e0542aaa489a6133205108e
I20260807 10:49:53.875834 6435 tablet_service.cc:1514] Processing DeleteTablet for tablet cc93601c0e0542aaa489a6133205108e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:49:53 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:53254
I20260807 10:49:53.876158 6946 tablet_replica.cc:330] T cc93601c0e0542aaa489a6133205108e P dd8042b5b9ce4a61a5ca306878f9ab7a: stopping tablet replica
I20260807 10:49:53.876099 6220 tablet_service.cc:1514] Processing DeleteTablet for tablet cc93601c0e0542aaa489a6133205108e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:49:53 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:42858
I20260807 10:49:53.876322 6640 tablet_service.cc:1514] Processing DeleteTablet for tablet cc93601c0e0542aaa489a6133205108e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:49:53 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:38242
I20260807 10:49:53.876390 6946 raft_consensus.cc:2229] T cc93601c0e0542aaa489a6133205108e P dd8042b5b9ce4a61a5ca306878f9ab7a [term 1 LEADER]: Raft consensus shutting down.
I20260807 10:49:53.876642 6947 tablet_replica.cc:330] T cc93601c0e0542aaa489a6133205108e P ea7e70332cf64545889d381b59a7d558: stopping tablet replica
I20260807 10:49:53.876832 6948 tablet_replica.cc:330] T cc93601c0e0542aaa489a6133205108e P 3519d7d61a6f4a50909174f45b035d9b: stopping tablet replica
I20260807 10:49:53.876859 6946 raft_consensus.cc:2258] T cc93601c0e0542aaa489a6133205108e P dd8042b5b9ce4a61a5ca306878f9ab7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:49:53.877031 6947 raft_consensus.cc:2229] T cc93601c0e0542aaa489a6133205108e P ea7e70332cf64545889d381b59a7d558 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:49:53.877197 6948 raft_consensus.cc:2229] T cc93601c0e0542aaa489a6133205108e P 3519d7d61a6f4a50909174f45b035d9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:49:53.877357 6947 raft_consensus.cc:2258] T cc93601c0e0542aaa489a6133205108e P ea7e70332cf64545889d381b59a7d558 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:49:53.877481 6948 raft_consensus.cc:2258] T cc93601c0e0542aaa489a6133205108e P 3519d7d61a6f4a50909174f45b035d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:49:53.877714 6010 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37418:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 10:49:53.878773 6947 ts_tablet_manager.cc:1905] T cc93601c0e0542aaa489a6133205108e P ea7e70332cf64545889d381b59a7d558: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 10:49:53.878786 6010 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 10:49:53.878897 6946 ts_tablet_manager.cc:1905] T cc93601c0e0542aaa489a6133205108e P dd8042b5b9ce4a61a5ca306878f9ab7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:49:53.879396 6948 ts_tablet_manager.cc:1905] T cc93601c0e0542aaa489a6133205108e P 3519d7d61a6f4a50909174f45b035d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:49:53.886734 6948 ts_tablet_manager.cc:1918] T cc93601c0e0542aaa489a6133205108e P 3519d7d61a6f4a50909174f45b035d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 10:49:53.887220 6948 log.cc:1196] T cc93601c0e0542aaa489a6133205108e P 3519d7d61a6f4a50909174f45b035d9b: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-2/wal/wals/cc93601c0e0542aaa489a6133205108e
I20260807 10:49:53.887214 6946 ts_tablet_manager.cc:1918] T cc93601c0e0542aaa489a6133205108e P dd8042b5b9ce4a61a5ca306878f9ab7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 10:49:53.887518 6946 log.cc:1196] T cc93601c0e0542aaa489a6133205108e P dd8042b5b9ce4a61a5ca306878f9ab7a: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-1/wal/wals/cc93601c0e0542aaa489a6133205108e
I20260807 10:49:53.888031 6946 ts_tablet_manager.cc:1939] T cc93601c0e0542aaa489a6133205108e P dd8042b5b9ce4a61a5ca306878f9ab7a: Deleting consensus metadata
I20260807 10:49:53.888032 6948 ts_tablet_manager.cc:1939] T cc93601c0e0542aaa489a6133205108e P 3519d7d61a6f4a50909174f45b035d9b: Deleting consensus metadata
I20260807 10:49:53.888921 6947 ts_tablet_manager.cc:1918] T cc93601c0e0542aaa489a6133205108e P ea7e70332cf64545889d381b59a7d558: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 10:49:53.889601 5984 catalog_manager.cc:4841] TS 3519d7d61a6f4a50909174f45b035d9b (127.3.73.195:39467): tablet cc93601c0e0542aaa489a6133205108e (table test-table [id=d5f4212c6db84f6ca2f4c082474f303b]) successfully deleted
I20260807 10:49:53.889855 6947 log.cc:1196] T cc93601c0e0542aaa489a6133205108e P ea7e70332cf64545889d381b59a7d558: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-0/wal/wals/cc93601c0e0542aaa489a6133205108e
I20260807 10:49:53.890761 6947 ts_tablet_manager.cc:1939] T cc93601c0e0542aaa489a6133205108e P ea7e70332cf64545889d381b59a7d558: Deleting consensus metadata
I20260807 10:49:53.890786 6220 tablet_service.cc:1467] Processing CreateTablet for tablet 6cec1bcd5243404586d330ca74d2d7c6 (DEFAULT_TABLE table=test-table [id=64a0d7914eeb47a9a52965b968b974ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:49:53.891250 6435 tablet_service.cc:1467] Processing CreateTablet for tablet 6cec1bcd5243404586d330ca74d2d7c6 (DEFAULT_TABLE table=test-table [id=64a0d7914eeb47a9a52965b968b974ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:49:53.891496 6220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6cec1bcd5243404586d330ca74d2d7c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:49:53.891695 6435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6cec1bcd5243404586d330ca74d2d7c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:49:53.892163 5986 catalog_manager.cc:4841] TS ea7e70332cf64545889d381b59a7d558 (127.3.73.193:44797): tablet cc93601c0e0542aaa489a6133205108e (table test-table [id=d5f4212c6db84f6ca2f4c082474f303b]) successfully deleted
I20260807 10:49:53.893093 5984 catalog_manager.cc:4841] TS dd8042b5b9ce4a61a5ca306878f9ab7a (127.3.73.194:41391): tablet cc93601c0e0542aaa489a6133205108e (table test-table [id=d5f4212c6db84f6ca2f4c082474f303b]) successfully deleted
I20260807 10:49:53.893246 6640 tablet_service.cc:1467] Processing CreateTablet for tablet 6cec1bcd5243404586d330ca74d2d7c6 (DEFAULT_TABLE table=test-table [id=64a0d7914eeb47a9a52965b968b974ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:49:53.893915 6640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6cec1bcd5243404586d330ca74d2d7c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:49:53.898934 6961 tablet_bootstrap.cc:492] T 6cec1bcd5243404586d330ca74d2d7c6 P dd8042b5b9ce4a61a5ca306878f9ab7a: Bootstrap starting.
I20260807 10:49:53.902249 6959 tablet_bootstrap.cc:492] T 6cec1bcd5243404586d330ca74d2d7c6 P ea7e70332cf64545889d381b59a7d558: Bootstrap starting.
I20260807 10:49:53.903404 6961 tablet_bootstrap.cc:654] T 6cec1bcd5243404586d330ca74d2d7c6 P dd8042b5b9ce4a61a5ca306878f9ab7a: Neither blocks nor log segments found. Creating new log.
I20260807 10:49:53.904345 6960 tablet_bootstrap.cc:492] T 6cec1bcd5243404586d330ca74d2d7c6 P 3519d7d61a6f4a50909174f45b035d9b: Bootstrap starting.
I20260807 10:49:53.905400 6959 tablet_bootstrap.cc:654] T 6cec1bcd5243404586d330ca74d2d7c6 P ea7e70332cf64545889d381b59a7d558: Neither blocks nor log segments found. Creating new log.
I20260807 10:49:53.909164 6961 tablet_bootstrap.cc:492] T 6cec1bcd5243404586d330ca74d2d7c6 P dd8042b5b9ce4a61a5ca306878f9ab7a: No bootstrap required, opened a new log
I20260807 10:49:53.909361 6960 tablet_bootstrap.cc:654] T 6cec1bcd5243404586d330ca74d2d7c6 P 3519d7d61a6f4a50909174f45b035d9b: Neither blocks nor log segments found. Creating new log.
I20260807 10:49:53.909556 6961 ts_tablet_manager.cc:1397] T 6cec1bcd5243404586d330ca74d2d7c6 P dd8042b5b9ce4a61a5ca306878f9ab7a: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260807 10:49:53.911309 6961 raft_consensus.cc:348] T 6cec1bcd5243404586d330ca74d2d7c6 P dd8042b5b9ce4a61a5ca306878f9ab7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:53.911731 6961 raft_consensus.cc:374] T 6cec1bcd5243404586d330ca74d2d7c6 P dd8042b5b9ce4a61a5ca306878f9ab7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:49:53.911952 6961 raft_consensus.cc:729] T 6cec1bcd5243404586d330ca74d2d7c6 P dd8042b5b9ce4a61a5ca306878f9ab7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd8042b5b9ce4a61a5ca306878f9ab7a, State: Initialized, Role: FOLLOWER
I20260807 10:49:53.912468 6961 consensus_queue.cc:260] T 6cec1bcd5243404586d330ca74d2d7c6 P dd8042b5b9ce4a61a5ca306878f9ab7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:53.913362 6959 tablet_bootstrap.cc:492] T 6cec1bcd5243404586d330ca74d2d7c6 P ea7e70332cf64545889d381b59a7d558: No bootstrap required, opened a new log
I20260807 10:49:53.913738 6959 ts_tablet_manager.cc:1397] T 6cec1bcd5243404586d330ca74d2d7c6 P ea7e70332cf64545889d381b59a7d558: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.003s
I20260807 10:49:53.914302 6961 ts_tablet_manager.cc:1428] T 6cec1bcd5243404586d330ca74d2d7c6 P dd8042b5b9ce4a61a5ca306878f9ab7a: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260807 10:49:53.915175 6959 raft_consensus.cc:348] T 6cec1bcd5243404586d330ca74d2d7c6 P ea7e70332cf64545889d381b59a7d558 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:53.915627 6960 tablet_bootstrap.cc:492] T 6cec1bcd5243404586d330ca74d2d7c6 P 3519d7d61a6f4a50909174f45b035d9b: No bootstrap required, opened a new log
I20260807 10:49:53.915625 6959 raft_consensus.cc:374] T 6cec1bcd5243404586d330ca74d2d7c6 P ea7e70332cf64545889d381b59a7d558 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:49:53.915802 6959 raft_consensus.cc:729] T 6cec1bcd5243404586d330ca74d2d7c6 P ea7e70332cf64545889d381b59a7d558 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea7e70332cf64545889d381b59a7d558, State: Initialized, Role: FOLLOWER
I20260807 10:49:53.916144 6960 ts_tablet_manager.cc:1397] T 6cec1bcd5243404586d330ca74d2d7c6 P 3519d7d61a6f4a50909174f45b035d9b: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.005s
I20260807 10:49:53.916420 6959 consensus_queue.cc:260] T 6cec1bcd5243404586d330ca74d2d7c6 P ea7e70332cf64545889d381b59a7d558 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:53.917557 6959 ts_tablet_manager.cc:1428] T 6cec1bcd5243404586d330ca74d2d7c6 P ea7e70332cf64545889d381b59a7d558: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260807 10:49:53.917946 6960 raft_consensus.cc:348] T 6cec1bcd5243404586d330ca74d2d7c6 P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:53.918357 6960 raft_consensus.cc:374] T 6cec1bcd5243404586d330ca74d2d7c6 P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:49:53.918597 6960 raft_consensus.cc:729] T 6cec1bcd5243404586d330ca74d2d7c6 P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3519d7d61a6f4a50909174f45b035d9b, State: Initialized, Role: FOLLOWER
I20260807 10:49:53.919102 6960 consensus_queue.cc:260] T 6cec1bcd5243404586d330ca74d2d7c6 P 3519d7d61a6f4a50909174f45b035d9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:53.919862 6960 ts_tablet_manager.cc:1428] T 6cec1bcd5243404586d330ca74d2d7c6 P 3519d7d61a6f4a50909174f45b035d9b: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 10:49:54.196465 6931 raft_consensus.cc:482] T 6cec1bcd5243404586d330ca74d2d7c6 P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 10:49:54.196933 6931 raft_consensus.cc:504] T 6cec1bcd5243404586d330ca74d2d7c6 P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:54.198504 6931 leader_election.cc:288] T 6cec1bcd5243404586d330ca74d2d7c6 P 3519d7d61a6f4a50909174f45b035d9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea7e70332cf64545889d381b59a7d558 (127.3.73.193:44797), dd8042b5b9ce4a61a5ca306878f9ab7a (127.3.73.194:41391)
I20260807 10:49:54.199299 6263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cec1bcd5243404586d330ca74d2d7c6" candidate_uuid: "3519d7d61a6f4a50909174f45b035d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea7e70332cf64545889d381b59a7d558" is_pre_election: true
I20260807 10:49:54.199553 6463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cec1bcd5243404586d330ca74d2d7c6" candidate_uuid: "3519d7d61a6f4a50909174f45b035d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" is_pre_election: true
I20260807 10:49:54.199831 6263 raft_consensus.cc:2454] T 6cec1bcd5243404586d330ca74d2d7c6 P ea7e70332cf64545889d381b59a7d558 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3519d7d61a6f4a50909174f45b035d9b in term 0.
I20260807 10:49:54.200062 6463 raft_consensus.cc:2454] T 6cec1bcd5243404586d330ca74d2d7c6 P dd8042b5b9ce4a61a5ca306878f9ab7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3519d7d61a6f4a50909174f45b035d9b in term 0.
I20260807 10:49:54.200462 6570 leader_election.cc:302] T 6cec1bcd5243404586d330ca74d2d7c6 P 3519d7d61a6f4a50909174f45b035d9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3519d7d61a6f4a50909174f45b035d9b, ea7e70332cf64545889d381b59a7d558; no voters:
I20260807 10:49:54.200937 6931 raft_consensus.cc:2781] T 6cec1bcd5243404586d330ca74d2d7c6 P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 10:49:54.201323 6931 raft_consensus.cc:482] T 6cec1bcd5243404586d330ca74d2d7c6 P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 10:49:54.201565 6931 raft_consensus.cc:3037] T 6cec1bcd5243404586d330ca74d2d7c6 P 3519d7d61a6f4a50909174f45b035d9b [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:49:54.205397 6931 raft_consensus.cc:504] T 6cec1bcd5243404586d330ca74d2d7c6 P 3519d7d61a6f4a50909174f45b035d9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:54.206641 6931 leader_election.cc:288] T 6cec1bcd5243404586d330ca74d2d7c6 P 3519d7d61a6f4a50909174f45b035d9b [CANDIDATE]: Term 1 election: Requested vote from peers ea7e70332cf64545889d381b59a7d558 (127.3.73.193:44797), dd8042b5b9ce4a61a5ca306878f9ab7a (127.3.73.194:41391)
I20260807 10:49:54.207362 6263 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cec1bcd5243404586d330ca74d2d7c6" candidate_uuid: "3519d7d61a6f4a50909174f45b035d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea7e70332cf64545889d381b59a7d558"
I20260807 10:49:54.207420 6463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6cec1bcd5243404586d330ca74d2d7c6" candidate_uuid: "3519d7d61a6f4a50909174f45b035d9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a"
I20260807 10:49:54.207764 6463 raft_consensus.cc:3037] T 6cec1bcd5243404586d330ca74d2d7c6 P dd8042b5b9ce4a61a5ca306878f9ab7a [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:49:54.207791 6263 raft_consensus.cc:3037] T 6cec1bcd5243404586d330ca74d2d7c6 P ea7e70332cf64545889d381b59a7d558 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:49:54.211165 6463 raft_consensus.cc:2454] T 6cec1bcd5243404586d330ca74d2d7c6 P dd8042b5b9ce4a61a5ca306878f9ab7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3519d7d61a6f4a50909174f45b035d9b in term 1.
I20260807 10:49:54.211803 6567 leader_election.cc:302] T 6cec1bcd5243404586d330ca74d2d7c6 P 3519d7d61a6f4a50909174f45b035d9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3519d7d61a6f4a50909174f45b035d9b, dd8042b5b9ce4a61a5ca306878f9ab7a; no voters:
I20260807 10:49:54.212316 6263 raft_consensus.cc:2454] T 6cec1bcd5243404586d330ca74d2d7c6 P ea7e70332cf64545889d381b59a7d558 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3519d7d61a6f4a50909174f45b035d9b in term 1.
I20260807 10:49:54.212358 6931 raft_consensus.cc:2781] T 6cec1bcd5243404586d330ca74d2d7c6 P 3519d7d61a6f4a50909174f45b035d9b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:49:54.212826 6931 raft_consensus.cc:686] T 6cec1bcd5243404586d330ca74d2d7c6 P 3519d7d61a6f4a50909174f45b035d9b [term 1 LEADER]: Becoming Leader. State: Replica: 3519d7d61a6f4a50909174f45b035d9b, State: Running, Role: LEADER
I20260807 10:49:54.213306 6931 consensus_queue.cc:237] T 6cec1bcd5243404586d330ca74d2d7c6 P 3519d7d61a6f4a50909174f45b035d9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } }
I20260807 10:49:54.218257 6008 catalog_manager.cc:5495] T 6cec1bcd5243404586d330ca74d2d7c6 P 3519d7d61a6f4a50909174f45b035d9b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3519d7d61a6f4a50909174f45b035d9b (127.3.73.195). New cstate: current_term: 1 leader_uuid: "3519d7d61a6f4a50909174f45b035d9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3519d7d61a6f4a50909174f45b035d9b" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39467 } health_report { overall_health: HEALTHY } } }
I20260807 10:49:54.298089 6991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:54.288441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35990 (local address 127.3.73.195:39467)
0807 10:49:54.289843 (+ 1402us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:54.289848 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:54.289891 (+ 43us) server_negotiation.cc:407] Connection header received
0807 10:49:54.289951 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:54.289955 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:54.290022 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:54.290118 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:49:54.291190 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:54.292910 (+ 1720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:54.295001 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:54.295887 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:54.296004 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:54.296841 (+ 837us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:49:54.297006 (+ 165us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:49:54.297095 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:54.297301 (+ 206us) server_negotiation.cc:299] Negotiation successful
0807 10:49:54.297403 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1238,"thread_start_us":970,"threads_started":1}
I20260807 10:49:54.311434 6463 raft_consensus.cc:1261] T 6cec1bcd5243404586d330ca74d2d7c6 P dd8042b5b9ce4a61a5ca306878f9ab7a [term 1 FOLLOWER]: Refusing update from remote peer 3519d7d61a6f4a50909174f45b035d9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:49:54.311882 6263 raft_consensus.cc:1261] T 6cec1bcd5243404586d330ca74d2d7c6 P ea7e70332cf64545889d381b59a7d558 [term 1 FOLLOWER]: Refusing update from remote peer 3519d7d61a6f4a50909174f45b035d9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:49:54.312258 6914 consensus_queue.cc:1035] T 6cec1bcd5243404586d330ca74d2d7c6 P 3519d7d61a6f4a50909174f45b035d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd8042b5b9ce4a61a5ca306878f9ab7a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 10:49:54.313038 6931 consensus_queue.cc:1035] T 6cec1bcd5243404586d330ca74d2d7c6 P 3519d7d61a6f4a50909174f45b035d9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea7e70332cf64545889d381b59a7d558" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 10:49:54.340413 6921 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 10:49:55.942022 6008 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:49:55.942404 6008 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:49:55.948534 6008 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8ea443e2ab0a47f1b65df7aeac9255e8: Sending DeleteTablet for 3 replicas of tablet 6cec1bcd5243404586d330ca74d2d7c6
I20260807 10:49:55.949805 6435 tablet_service.cc:1514] Processing DeleteTablet for tablet 6cec1bcd5243404586d330ca74d2d7c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:49:55 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:53254
I20260807 10:49:55.949904 6220 tablet_service.cc:1514] Processing DeleteTablet for tablet 6cec1bcd5243404586d330ca74d2d7c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:49:55 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:42858
I20260807 10:49:55.950086 6640 tablet_service.cc:1514] Processing DeleteTablet for tablet 6cec1bcd5243404586d330ca74d2d7c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:49:55 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:38242
I20260807 10:49:55.953557 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 6118
I20260807 10:49:55.968430 7040 tablet_replica.cc:330] T 6cec1bcd5243404586d330ca74d2d7c6 P 3519d7d61a6f4a50909174f45b035d9b: stopping tablet replica
I20260807 10:49:55.968842 7040 raft_consensus.cc:2229] T 6cec1bcd5243404586d330ca74d2d7c6 P 3519d7d61a6f4a50909174f45b035d9b [term 1 LEADER]: Raft consensus shutting down.
I20260807 10:49:55.969597 7040 raft_consensus.cc:2258] T 6cec1bcd5243404586d330ca74d2d7c6 P 3519d7d61a6f4a50909174f45b035d9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:49:55.971220 7040 ts_tablet_manager.cc:1905] T 6cec1bcd5243404586d330ca74d2d7c6 P 3519d7d61a6f4a50909174f45b035d9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:49:55.981060 7040 ts_tablet_manager.cc:1918] T 6cec1bcd5243404586d330ca74d2d7c6 P 3519d7d61a6f4a50909174f45b035d9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 10:49:55.981299 7038 tablet_replica.cc:330] T 6cec1bcd5243404586d330ca74d2d7c6 P dd8042b5b9ce4a61a5ca306878f9ab7a: stopping tablet replica
I20260807 10:49:55.981451 7040 log.cc:1196] T 6cec1bcd5243404586d330ca74d2d7c6 P 3519d7d61a6f4a50909174f45b035d9b: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-2/wal/wals/6cec1bcd5243404586d330ca74d2d7c6
I20260807 10:49:55.981612 7038 raft_consensus.cc:2229] T 6cec1bcd5243404586d330ca74d2d7c6 P dd8042b5b9ce4a61a5ca306878f9ab7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:49:55.982136 7038 raft_consensus.cc:2258] T 6cec1bcd5243404586d330ca74d2d7c6 P dd8042b5b9ce4a61a5ca306878f9ab7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:49:55.982367 7040 ts_tablet_manager.cc:1939] T 6cec1bcd5243404586d330ca74d2d7c6 P 3519d7d61a6f4a50909174f45b035d9b: Deleting consensus metadata
I20260807 10:49:55.983166 7037 tablet_replica.cc:330] T 6cec1bcd5243404586d330ca74d2d7c6 P ea7e70332cf64545889d381b59a7d558: stopping tablet replica
I20260807 10:49:55.983474 7037 raft_consensus.cc:2229] T 6cec1bcd5243404586d330ca74d2d7c6 P ea7e70332cf64545889d381b59a7d558 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:49:55.984030 7037 raft_consensus.cc:2258] T 6cec1bcd5243404586d330ca74d2d7c6 P ea7e70332cf64545889d381b59a7d558 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:49:55.984031 5984 catalog_manager.cc:4841] TS 3519d7d61a6f4a50909174f45b035d9b (127.3.73.195:39467): tablet 6cec1bcd5243404586d330ca74d2d7c6 (table test-table [id=64a0d7914eeb47a9a52965b968b974ae]) successfully deleted
I20260807 10:49:55.985117 7038 ts_tablet_manager.cc:1905] T 6cec1bcd5243404586d330ca74d2d7c6 P dd8042b5b9ce4a61a5ca306878f9ab7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:49:55.985306 7037 ts_tablet_manager.cc:1905] T 6cec1bcd5243404586d330ca74d2d7c6 P ea7e70332cf64545889d381b59a7d558: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 10:49:55 dist-test-slave-x5fp krb5kdc[5938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099793, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.73.193@KRBTEST.COM
I20260807 10:49:55.994751 7038 ts_tablet_manager.cc:1918] T 6cec1bcd5243404586d330ca74d2d7c6 P dd8042b5b9ce4a61a5ca306878f9ab7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 10:49:55.994963 7038 log.cc:1196] T 6cec1bcd5243404586d330ca74d2d7c6 P dd8042b5b9ce4a61a5ca306878f9ab7a: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-1/wal/wals/6cec1bcd5243404586d330ca74d2d7c6
I20260807 10:49:55.995321 7037 ts_tablet_manager.cc:1918] T 6cec1bcd5243404586d330ca74d2d7c6 P ea7e70332cf64545889d381b59a7d558: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 10:49:55.995695 7037 log.cc:1196] T 6cec1bcd5243404586d330ca74d2d7c6 P ea7e70332cf64545889d381b59a7d558: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786099767749326-3367-0/minicluster-data/ts-0/wal/wals/6cec1bcd5243404586d330ca74d2d7c6
I20260807 10:49:55.995838 7038 ts_tablet_manager.cc:1939] T 6cec1bcd5243404586d330ca74d2d7c6 P dd8042b5b9ce4a61a5ca306878f9ab7a: Deleting consensus metadata
I20260807 10:49:55.996790 7037 ts_tablet_manager.cc:1939] T 6cec1bcd5243404586d330ca74d2d7c6 P ea7e70332cf64545889d381b59a7d558: Deleting consensus metadata
I20260807 10:49:55.997486 5984 catalog_manager.cc:4841] TS dd8042b5b9ce4a61a5ca306878f9ab7a (127.3.73.194:41391): tablet 6cec1bcd5243404586d330ca74d2d7c6 (table test-table [id=64a0d7914eeb47a9a52965b968b974ae]) successfully deleted
I20260807 10:49:55.998113 5986 catalog_manager.cc:4841] TS ea7e70332cf64545889d381b59a7d558 (127.3.73.193:44797): tablet 6cec1bcd5243404586d330ca74d2d7c6 (table test-table [id=64a0d7914eeb47a9a52965b968b974ae]) successfully deleted
I20260807 10:49:56.000267 7042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:55.955780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59554 (local address 127.3.73.193:44797)
0807 10:49:55.973456 (+ 17676us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:55.973464 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:55.974460 (+ 996us) server_negotiation.cc:407] Connection header received
0807 10:49:55.974568 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:55.974574 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:55.974706 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:55.974846 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:55.977887 (+ 3041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:55.980079 (+ 2192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:55.983817 (+ 3738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:55.984564 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:55.984773 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:55.990828 (+ 6055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:55.990845 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:55.990862 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:55.990903 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:55.996296 (+ 5393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:55.997332 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:55.997346 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:55.997352 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:55.997577 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:55.998146 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:55.998149 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:55.998152 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:55.998769 (+ 617us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:55.999016 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:55.999459 (+ 443us) server_negotiation.cc:299] Negotiation successful
0807 10:49:55.999695 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17447,"thread_start_us":17233,"threads_started":1}
W20260807 10:49:56.001305 6323 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59554
W20260807 10:49:56.002135 3367 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 6118: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 10:49:56.002490 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 6118
I20260807 10:49:56.035948 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 6334
Aug 07 10:49:56 dist-test-slave-x5fp krb5kdc[5938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099793, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.73.194@KRBTEST.COM
I20260807 10:49:56.054368 7043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:56.036820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37660 (local address 127.3.73.194:41391)
0807 10:49:56.037497 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:56.037502 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:56.037520 (+ 18us) server_negotiation.cc:407] Connection header received
0807 10:49:56.037578 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:56.037582 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:56.037650 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:56.037745 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:56.039229 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:56.040020 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:56.041641 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:56.042075 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:56.042179 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:56.046924 (+ 4745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:56.046941 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:56.046959 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:56.047031 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:56.050521 (+ 3490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:56.051368 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:56.051376 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:56.051379 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:56.051541 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:56.052002 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:56.052006 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:56.052009 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:56.052845 (+ 836us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:56.053239 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:56.053692 (+ 453us) server_negotiation.cc:299] Negotiation successful
0807 10:49:56.053985 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":324,"threads_started":1}
W20260807 10:49:56.055132 6532 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37660
W20260807 10:49:56.055867 3367 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 6334: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 10:49:56.056023 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 6334
I20260807 10:49:56.091615 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 6543
Aug 07 10:49:56 dist-test-slave-x5fp krb5kdc[5938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099793, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.73.195@KRBTEST.COM
I20260807 10:49:56.111915 7044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:56.092622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35992 (local address 127.3.73.195:39467)
0807 10:49:56.093289 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:56.093295 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:56.093315 (+ 20us) server_negotiation.cc:407] Connection header received
0807 10:49:56.093375 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:56.093382 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:56.093463 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:56.093587 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:56.094990 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:56.096178 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:56.097631 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:56.098328 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:56.098451 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:56.103506 (+ 5055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:56.103539 (+ 33us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:56.103563 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:56.103607 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:56.108048 (+ 4441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:56.109240 (+ 1192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:56.109252 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:56.109255 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:56.109522 (+ 267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:56.109976 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:56.109983 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:56.109988 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:56.110475 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:56.110763 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:56.111088 (+ 325us) server_negotiation.cc:299] Negotiation successful
0807 10:49:56.111389 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":305,"threads_started":1}
W20260807 10:49:56.112804 6736 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35992
W20260807 10:49:56.113530 3367 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 6543: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 10:49:56.113704 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 6543
I20260807 10:49:56.154450 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 5959
I20260807 10:49:56.155990 6073 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:49:56.321403 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 5959
2026-08-07T10:49:56Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (11172 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:49:56 dist-test-slave-x5fp krb5kdc[7053](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:49:56 dist-test-slave-x5fp krb5kdc[7053](info): set up 2 sockets
Aug 07 10:49:56 dist-test-slave-x5fp krb5kdc[7053](info): commencing operation
krb5kdc: starting...
W20260807 10:49:57.763897 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.388s user 0.007s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:49:57 dist-test-slave-x5fp krb5kdc[7053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:49:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:49:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.73.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
I20260807 10:49:58.038467 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:43841
--webserver_interface=127.3.73.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:34495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:43841
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:49:58.416169 7074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:49:58.416499 7074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:49:58.416697 7074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:49:58.425006 7074 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:49:58.425307 7074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:49:58.425434 7074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:49:58.425545 7074 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:49:58.425635 7074 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:49:58.436345 7074 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:34495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:43841
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:43841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:49:58.437922 7074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:49:58.439384 7074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:49:58.453434 7082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:58.453490 7086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:58.453461 7083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:49:58.453934 7074 server_base.cc:1034] running on GCE node
I20260807 10:49:58.455047 7074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:49:58.456743 7074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:49:58.458047 7074 hybrid_clock.cc:648] HybridClock initialized: now 1786099798457984 us; error 36 us; skew 500 ppm
I20260807 10:49:58.458441 7074 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:49:58 dist-test-slave-x5fp krb5kdc[7053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099798, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:49:58.474244 7074 init.cc:377] Logged in from keytab as kudu/127.3.73.254@KRBTEST.COM (short username kudu)
I20260807 10:49:58.476431 7074 webserver.cc:460] Webserver started at http://127.3.73.254:33991/ using document root <none> and password file <none>
I20260807 10:49:58.477320 7074 fs_manager.cc:359] Metadata directory not provided
I20260807 10:49:58.477474 7074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:49:58.477942 7074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:49:58.481664 7074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "77a0b0ffa32b48d6b522bba0166a6576"
format_stamp: "Formatted at 2026-08-07 10:49:58 on dist-test-slave-x5fp"
server_key: "8f8fa0555baf95497219918b8ec2c470"
server_key_iv: "4a6fc46417af64f3c10e4b3273d6dba8"
server_key_version: "encryptionkey@0"
I20260807 10:49:58.482606 7074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "77a0b0ffa32b48d6b522bba0166a6576"
format_stamp: "Formatted at 2026-08-07 10:49:58 on dist-test-slave-x5fp"
server_key: "8f8fa0555baf95497219918b8ec2c470"
server_key_iv: "4a6fc46417af64f3c10e4b3273d6dba8"
server_key_version: "encryptionkey@0"
I20260807 10:49:58.488993 7074 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260807 10:49:58.493359 7097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:58.495126 7074 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 10:49:58.495369 7074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "77a0b0ffa32b48d6b522bba0166a6576"
format_stamp: "Formatted at 2026-08-07 10:49:58 on dist-test-slave-x5fp"
server_key: "8f8fa0555baf95497219918b8ec2c470"
server_key_iv: "4a6fc46417af64f3c10e4b3273d6dba8"
server_key_version: "encryptionkey@0"
I20260807 10:49:58.495643 7074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:49:58.512986 7074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:49:58.518169 7074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:49:58.518550 7074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:49:58.542429 7074 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:43841
I20260807 10:49:58.542440 7182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:43841 every 8 connection(s)
I20260807 10:49:58.544407 7074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:49:58.546792 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 7074
I20260807 10:49:58.547066 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:49:58.547995 3367 external_mini_cluster.cc:1428] Setting key a5a58a7f7185bf635833bba1a4e8ee5a
I20260807 10:49:58.550424 7184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:49:58.563755 7184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77a0b0ffa32b48d6b522bba0166a6576: Bootstrap starting.
Aug 07 10:49:58 dist-test-slave-x5fp krb5kdc[7053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:49:58.568706 7184 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77a0b0ffa32b48d6b522bba0166a6576: Neither blocks nor log segments found. Creating new log.
I20260807 10:49:58.570135 7184 log.cc:824] T 00000000000000000000000000000000 P 77a0b0ffa32b48d6b522bba0166a6576: Log is configured to *not* fsync() on all Append() calls
I20260807 10:49:58.574967 7184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77a0b0ffa32b48d6b522bba0166a6576: No bootstrap required, opened a new log
I20260807 10:49:58.575517 7187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:58.550561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48772 (local address 127.3.73.254:43841)
0807 10:49:58.551570 (+ 1009us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:58.551584 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:58.551629 (+ 45us) server_negotiation.cc:407] Connection header received
0807 10:49:58.553687 (+ 2058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:58.553714 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:58.554175 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:58.554677 (+ 502us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:58.556407 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:58.557958 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:58.559464 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:58.560259 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:58.560524 (+ 265us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:58.565245 (+ 4721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:58.565254 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:58.565270 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:58.565316 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:58.569565 (+ 4249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:58.570545 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:58.570556 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:58.570571 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:58.570878 (+ 307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:58.571564 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:58.571569 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:58.571571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:58.572337 (+ 766us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:58.572705 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:58.573361 (+ 656us) server_negotiation.cc:299] Negotiation successful
0807 10:49:58.573890 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":770,"thread_start_us":475,"threads_started":1}
I20260807 10:49:58.584879 7184 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77a0b0ffa32b48d6b522bba0166a6576 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77a0b0ffa32b48d6b522bba0166a6576" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 43841 } }
I20260807 10:49:58.585382 7184 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77a0b0ffa32b48d6b522bba0166a6576 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:49:58.585522 7184 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77a0b0ffa32b48d6b522bba0166a6576 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77a0b0ffa32b48d6b522bba0166a6576, State: Initialized, Role: FOLLOWER
I20260807 10:49:58.586306 7184 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77a0b0ffa32b48d6b522bba0166a6576 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77a0b0ffa32b48d6b522bba0166a6576" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 43841 } }
I20260807 10:49:58.586649 7184 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77a0b0ffa32b48d6b522bba0166a6576 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:49:58.586815 7184 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77a0b0ffa32b48d6b522bba0166a6576 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:49:58.587019 7184 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77a0b0ffa32b48d6b522bba0166a6576 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:49:58.589534 7184 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77a0b0ffa32b48d6b522bba0166a6576 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77a0b0ffa32b48d6b522bba0166a6576" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 43841 } }
I20260807 10:49:58.590147 7184 leader_election.cc:302] T 00000000000000000000000000000000 P 77a0b0ffa32b48d6b522bba0166a6576 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77a0b0ffa32b48d6b522bba0166a6576; no voters:
I20260807 10:49:58.591013 7184 leader_election.cc:288] T 00000000000000000000000000000000 P 77a0b0ffa32b48d6b522bba0166a6576 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:49:58.591078 7189 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77a0b0ffa32b48d6b522bba0166a6576 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:49:58.592096 7189 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77a0b0ffa32b48d6b522bba0166a6576 [term 1 LEADER]: Becoming Leader. State: Replica: 77a0b0ffa32b48d6b522bba0166a6576, State: Running, Role: LEADER
I20260807 10:49:58.592935 7189 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77a0b0ffa32b48d6b522bba0166a6576 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77a0b0ffa32b48d6b522bba0166a6576" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 43841 } }
I20260807 10:49:58.594205 7184 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77a0b0ffa32b48d6b522bba0166a6576 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:49:58.596588 7191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77a0b0ffa32b48d6b522bba0166a6576 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77a0b0ffa32b48d6b522bba0166a6576" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77a0b0ffa32b48d6b522bba0166a6576" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 43841 } } }
I20260807 10:49:58.596470 7190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77a0b0ffa32b48d6b522bba0166a6576 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77a0b0ffa32b48d6b522bba0166a6576. Latest consensus state: current_term: 1 leader_uuid: "77a0b0ffa32b48d6b522bba0166a6576" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77a0b0ffa32b48d6b522bba0166a6576" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 43841 } } }
I20260807 10:49:58.597381 7190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77a0b0ffa32b48d6b522bba0166a6576 [sys.catalog]: This master's current role is: LEADER
I20260807 10:49:58.597470 7191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77a0b0ffa32b48d6b522bba0166a6576 [sys.catalog]: This master's current role is: LEADER
I20260807 10:49:58.601239 7199 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:49:58.607710 7199 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:49:58.618870 7199 catalog_manager.cc:1347] Generated new cluster ID: 169dc4114fef42068279466e0d769479
I20260807 10:49:58.619050 7199 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:49:58.648197 7199 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:49:58.650071 7199 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:49:58.658174 7199 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77a0b0ffa32b48d6b522bba0166a6576: Generated new TSK 0
I20260807 10:49:58.659108 7199 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.73.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab.
Entry for principal kudu/127.3.73.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab.
I20260807 10:49:58.740137 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.193:0
--local_ip_for_outbound_sockets=127.3.73.193
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:43841
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:34495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:49:59.142242 7220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:49:59.142589 7220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:49:59.142781 7220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:49:59.150879 7220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:49:59.151024 7220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:49:59.151233 7220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.193
I20260807 10:49:59.162326 7220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:34495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:43841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.193
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:49:59.163968 7220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:49:59.165657 7220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:49:59.181124 7226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:59.181249 7225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:59.181489 7229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:49:59.182639 7220 server_base.cc:1034] running on GCE node
I20260807 10:49:59.183493 7220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:49:59.184592 7220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:49:59.185909 7220 hybrid_clock.cc:648] HybridClock initialized: now 1786099799185832 us; error 64 us; skew 500 ppm
I20260807 10:49:59.186280 7220 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:49:59 dist-test-slave-x5fp krb5kdc[7053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099799, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:49:59.190092 7220 init.cc:377] Logged in from keytab as kudu/127.3.73.193@KRBTEST.COM (short username kudu)
I20260807 10:49:59.191967 7220 webserver.cc:460] Webserver started at http://127.3.73.193:43763/ using document root <none> and password file <none>
I20260807 10:49:59.192909 7220 fs_manager.cc:359] Metadata directory not provided
I20260807 10:49:59.193077 7220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:49:59.193523 7220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:49:59.197482 7220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-0/data/instance:
uuid: "bfde6f7e039942f9bf58bdf65fca7d1d"
format_stamp: "Formatted at 2026-08-07 10:49:59 on dist-test-slave-x5fp"
server_key: "5af5b3d55b2a4915d1676f9697b32e2a"
server_key_iv: "42e66e69e4a34ee594480d384cbb2446"
server_key_version: "encryptionkey@0"
I20260807 10:49:59.198439 7220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance:
uuid: "bfde6f7e039942f9bf58bdf65fca7d1d"
format_stamp: "Formatted at 2026-08-07 10:49:59 on dist-test-slave-x5fp"
server_key: "5af5b3d55b2a4915d1676f9697b32e2a"
server_key_iv: "42e66e69e4a34ee594480d384cbb2446"
server_key_version: "encryptionkey@0"
I20260807 10:49:59.205406 7220 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 10:49:59.210110 7237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:59.212105 7220 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 10:49:59.212306 7220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-0/wal
uuid: "bfde6f7e039942f9bf58bdf65fca7d1d"
format_stamp: "Formatted at 2026-08-07 10:49:59 on dist-test-slave-x5fp"
server_key: "5af5b3d55b2a4915d1676f9697b32e2a"
server_key_iv: "42e66e69e4a34ee594480d384cbb2446"
server_key_version: "encryptionkey@0"
I20260807 10:49:59.212661 7220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:49:59.238996 7220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:49:59.244521 7220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:49:59.244958 7220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:49:59.246256 7220 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:49:59.248039 7220 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:49:59.248131 7220 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:59.248383 7220 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:49:59.248471 7220 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:59.295555 7220 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.193:44167
I20260807 10:49:59.295604 7415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.193:44167 every 8 connection(s)
I20260807 10:49:59.297083 7220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
I20260807 10:49:59.300987 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 7220
I20260807 10:49:59.301252 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance
I20260807 10:49:59.302167 3367 external_mini_cluster.cc:1428] Setting key 70df99ff7100633ffb4d45bcbd990400
Aug 07 10:49:59 dist-test-slave-x5fp krb5kdc[7053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099799, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:49:59.322471 7418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:59.300227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:52253 (local address 127.3.73.254:43841)
0807 10:49:59.300831 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:59.300835 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:59.302238 (+ 1403us) server_negotiation.cc:407] Connection header received
0807 10:49:59.304454 (+ 2216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:59.304460 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:59.304553 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:59.304708 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:59.306478 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:59.307264 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:59.309094 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:59.309571 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:59.309771 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:59.314071 (+ 4300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:59.314077 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:59.314081 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:59.314120 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:59.318014 (+ 3894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:59.318948 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:59.318957 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:59.318965 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:59.319244 (+ 279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:59.319851 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:59.319860 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:59.319867 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:59.320347 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:59.320540 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:59.321519 (+ 979us) server_negotiation.cc:299] Negotiation successful
0807 10:49:59.321849 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":290,"threads_started":1}
I20260807 10:49:59.323017 7419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:59.300920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:43841 (local address 127.3.73.193:52253)
0807 10:49:59.302014 (+ 1094us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:59.302070 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:59.304150 (+ 2080us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:59.305016 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:59.305025 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:59.305626 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:49:59.306277 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:59.306296 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:59.307453 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:59.307459 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:59.308930 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:59.308944 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:59.309881 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:59.309892 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:59.310216 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:59.311046 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:49:59.311075 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:49:59.313790 (+ 2715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:49:59.318225 (+ 4435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:59.318234 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:59.318247 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:59.318770 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:59.319439 (+ 669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:59.319444 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:59.319447 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:59.319701 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:59.320520 (+ 819us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:49:59.320527 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:49:59.321030 (+ 503us) client_negotiation.cc:769] Sending connection context
0807 10:49:59.321318 (+ 288us) client_negotiation.cc:240] Negotiation successful
0807 10:49:59.321762 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":813,"thread_start_us":491,"threads_started":1}
I20260807 10:49:59.324990 7416 heartbeater.cc:347] Connected to a master server at 127.3.73.254:43841
I20260807 10:49:59.325395 7416 heartbeater.cc:464] Registering TS with master...
I20260807 10:49:59.326558 7416 heartbeater.cc:511] Master 127.3.73.254:43841 requested a full tablet report, sending...
I20260807 10:49:59.329499 7121 ts_manager.cc:194] Registered new tserver with Master: bfde6f7e039942f9bf58bdf65fca7d1d (127.3.73.193:44167)
I20260807 10:49:59.331073 7121 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.193@KRBTEST.COM'} at 127.3.73.193:52253
WARNING: no policy specified for kudu/127.3.73.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab.
Entry for principal kudu/127.3.73.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab.
I20260807 10:49:59.369099 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.194:0
--local_ip_for_outbound_sockets=127.3.73.194
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:43841
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:34495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:49:59.774503 7422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:49:59.774978 7422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:49:59.775192 7422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:49:59.784093 7422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:49:59.784303 7422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:49:59.784509 7422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.194
I20260807 10:49:59.795141 7422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:34495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:43841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.194
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:49:59.796708 7422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:49:59.798174 7422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:49:59.813581 7435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:59.813522 7431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:49:59.813553 7429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:49:59.814368 7422 server_base.cc:1034] running on GCE node
I20260807 10:49:59.815130 7422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:49:59.816258 7422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:49:59.817551 7422 hybrid_clock.cc:648] HybridClock initialized: now 1786099799817498 us; error 45 us; skew 500 ppm
I20260807 10:49:59.817977 7422 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:49:59 dist-test-slave-x5fp krb5kdc[7053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099799, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:49:59.822098 7422 init.cc:377] Logged in from keytab as kudu/127.3.73.194@KRBTEST.COM (short username kudu)
I20260807 10:49:59.823886 7422 webserver.cc:460] Webserver started at http://127.3.73.194:37357/ using document root <none> and password file <none>
I20260807 10:49:59.824693 7422 fs_manager.cc:359] Metadata directory not provided
I20260807 10:49:59.824909 7422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:49:59.825222 7422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:49:59.828616 7422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-1/data/instance:
uuid: "08e1c3f232ff4537b6a74a2fe2f042b2"
format_stamp: "Formatted at 2026-08-07 10:49:59 on dist-test-slave-x5fp"
server_key: "1229e3d8d83a64c9069ca499f14aafac"
server_key_iv: "880b3b60a7bce82d30a2850e034f0924"
server_key_version: "encryptionkey@0"
I20260807 10:49:59.829589 7422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance:
uuid: "08e1c3f232ff4537b6a74a2fe2f042b2"
format_stamp: "Formatted at 2026-08-07 10:49:59 on dist-test-slave-x5fp"
server_key: "1229e3d8d83a64c9069ca499f14aafac"
server_key_iv: "880b3b60a7bce82d30a2850e034f0924"
server_key_version: "encryptionkey@0"
I20260807 10:49:59.835587 7422 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 10:49:59.839794 7448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:59.841601 7422 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 10:49:59.841774 7422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-1/wal
uuid: "08e1c3f232ff4537b6a74a2fe2f042b2"
format_stamp: "Formatted at 2026-08-07 10:49:59 on dist-test-slave-x5fp"
server_key: "1229e3d8d83a64c9069ca499f14aafac"
server_key_iv: "880b3b60a7bce82d30a2850e034f0924"
server_key_version: "encryptionkey@0"
I20260807 10:49:59.841979 7422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:49:59.868249 7422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:49:59.873795 7422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:49:59.874084 7422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:49:59.875272 7422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:49:59.877084 7422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:49:59.877177 7422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:59.877286 7422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:49:59.877344 7422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:49:59.921401 7422 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.194:37945
I20260807 10:49:59.921514 7612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.194:37945 every 8 connection(s)
I20260807 10:49:59.923007 7422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
I20260807 10:49:59.929539 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 7422
I20260807 10:49:59.929785 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance
I20260807 10:49:59.930475 3367 external_mini_cluster.cc:1428] Setting key 3803c9f2f2104ee32cb68eb3db608586
Aug 07 10:49:59 dist-test-slave-x5fp krb5kdc[7053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099799, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:49:59.949499 7615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:59.926145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:33097 (local address 127.3.73.254:43841)
0807 10:49:59.926654 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0807 10:49:59.926660 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:49:59.928088 (+ 1428us) server_negotiation.cc:407] Connection header received
0807 10:49:59.930236 (+ 2148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:49:59.930243 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:49:59.930353 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:49:59.930541 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:49:59.933075 (+ 2534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:59.934178 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:59.936001 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:49:59.936830 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:49:59.936986 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:59.941466 (+ 4480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:49:59.941476 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:49:59.941484 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:49:59.941539 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:49:59.945224 (+ 3685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:59.946599 (+ 1375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:59.946604 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:59.946607 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:59.946781 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:49:59.947396 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:49:59.947400 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:49:59.947402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:49:59.947789 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:49:59.947971 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 10:49:59.948804 (+ 833us) server_negotiation.cc:299] Negotiation successful
0807 10:49:59.949069 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":242,"threads_started":1}
I20260807 10:49:59.950666 7616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:49:59.926778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:43841 (local address 127.3.73.194:33097)
0807 10:49:59.927876 (+ 1098us) negotiation.cc:107] Waiting for socket to connect
0807 10:49:59.927924 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 10:49:59.929897 (+ 1973us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:49:59.930923 (+ 1026us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:49:59.930934 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:49:59.931766 (+ 832us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:49:59.932820 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:59.932851 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:59.934369 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:59.934374 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:59.935819 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:49:59.935834 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:49:59.937078 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:49:59.937083 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:49:59.937314 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:49:59.938178 (+ 864us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:49:59.938202 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:49:59.941127 (+ 2925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:49:59.945439 (+ 4312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:59.945455 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:59.945482 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:59.946365 (+ 883us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:59.946965 (+ 600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:49:59.946971 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:49:59.946975 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:49:59.947220 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:49:59.947954 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:49:59.947964 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:49:59.948358 (+ 394us) client_negotiation.cc:769] Sending connection context
0807 10:49:59.948673 (+ 315us) client_negotiation.cc:240] Negotiation successful
0807 10:49:59.949329 (+ 656us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":846,"thread_start_us":614,"threads_started":1}
I20260807 10:49:59.952453 7613 heartbeater.cc:347] Connected to a master server at 127.3.73.254:43841
I20260807 10:49:59.952864 7613 heartbeater.cc:464] Registering TS with master...
I20260807 10:49:59.953727 7613 heartbeater.cc:511] Master 127.3.73.254:43841 requested a full tablet report, sending...
I20260807 10:49:59.955742 7121 ts_manager.cc:194] Registered new tserver with Master: 08e1c3f232ff4537b6a74a2fe2f042b2 (127.3.73.194:37945)
I20260807 10:49:59.957043 7121 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.194@KRBTEST.COM'} at 127.3.73.194:33097
WARNING: no policy specified for kudu/127.3.73.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab.
Entry for principal kudu/127.3.73.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab.
I20260807 10:50:00.003124 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.195:0
--local_ip_for_outbound_sockets=127.3.73.195
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:43841
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:34495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 10:50:00.334686 7416 heartbeater.cc:503] Master 127.3.73.254:43841 was elected leader, sending a full tablet report...
W20260807 10:50:00.392361 7620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:00.392788 7620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:00.392979 7620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:00.400978 7620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:00.401187 7620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:00.401374 7620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.195
I20260807 10:50:00.412513 7620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:34495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:43841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.195
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:00.414089 7620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:00.415601 7620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:00.431579 7628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:00.431597 7632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:00.431622 7629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:00.432504 7620 server_base.cc:1034] running on GCE node
I20260807 10:50:00.433225 7620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:00.434286 7620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:00.435559 7620 hybrid_clock.cc:648] HybridClock initialized: now 1786099800435507 us; error 35 us; skew 500 ppm
I20260807 10:50:00.435971 7620 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:00 dist-test-slave-x5fp krb5kdc[7053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099800, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:00.439740 7620 init.cc:377] Logged in from keytab as kudu/127.3.73.195@KRBTEST.COM (short username kudu)
I20260807 10:50:00.441730 7620 webserver.cc:460] Webserver started at http://127.3.73.195:39403/ using document root <none> and password file <none>
I20260807 10:50:00.442556 7620 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:00.442744 7620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:00.443135 7620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:00.446714 7620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-2/data/instance:
uuid: "7bc6a60625ea42228ffa6e523fb12fd5"
format_stamp: "Formatted at 2026-08-07 10:50:00 on dist-test-slave-x5fp"
server_key: "14282194bd796b6126411c981fafdb8e"
server_key_iv: "0a5840c5767e6c989fac61d55e55d344"
server_key_version: "encryptionkey@0"
I20260807 10:50:00.447710 7620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance:
uuid: "7bc6a60625ea42228ffa6e523fb12fd5"
format_stamp: "Formatted at 2026-08-07 10:50:00 on dist-test-slave-x5fp"
server_key: "14282194bd796b6126411c981fafdb8e"
server_key_iv: "0a5840c5767e6c989fac61d55e55d344"
server_key_version: "encryptionkey@0"
I20260807 10:50:00.454600 7620 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260807 10:50:00.459000 7642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:00.461246 7620 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 10:50:00.461483 7620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-2/wal
uuid: "7bc6a60625ea42228ffa6e523fb12fd5"
format_stamp: "Formatted at 2026-08-07 10:50:00 on dist-test-slave-x5fp"
server_key: "14282194bd796b6126411c981fafdb8e"
server_key_iv: "0a5840c5767e6c989fac61d55e55d344"
server_key_version: "encryptionkey@0"
I20260807 10:50:00.461738 7620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:00.480607 7620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:00.486323 7620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:00.486620 7620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:00.488039 7620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:50:00.489863 7620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:50:00.489959 7620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:00.490065 7620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:50:00.490114 7620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:00.532836 7620 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.195:40667
I20260807 10:50:00.532867 7804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.195:40667 every 8 connection(s)
I20260807 10:50:00.534315 7620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
I20260807 10:50:00.544216 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 7620
I20260807 10:50:00.544498 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance
I20260807 10:50:00.545425 3367 external_mini_cluster.cc:1428] Setting key 3e020bbe9753414b0c6b36b23585f1a4
Aug 07 10:50:00 dist-test-slave-x5fp krb5kdc[7053](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099800, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:50:00.561077 7807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:00.537472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:48393 (local address 127.3.73.254:43841)
0807 10:50:00.538074 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:00.538080 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:00.539057 (+ 977us) server_negotiation.cc:407] Connection header received
0807 10:50:00.541303 (+ 2246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:00.541310 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:00.541486 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:00.541662 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:00.543554 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:00.545148 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:00.547769 (+ 2621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:00.548498 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:00.548629 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:00.553130 (+ 4501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:00.553137 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:00.553141 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:00.553262 (+ 121us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:00.556558 (+ 3296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:00.557720 (+ 1162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:00.557724 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:00.557727 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:00.558042 (+ 315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:00.558711 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:00.558716 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:00.558718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:00.559089 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:00.559231 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:00.560248 (+ 1017us) server_negotiation.cc:299] Negotiation successful
0807 10:50:00.560512 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":301,"threads_started":1}
I20260807 10:50:00.562947 7808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:00.537867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:43841 (local address 127.3.73.195:48393)
0807 10:50:00.538812 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:00.538862 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:00.540957 (+ 2095us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:00.541979 (+ 1022us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:00.541990 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:00.542619 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:00.543329 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:00.543350 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:00.545414 (+ 2064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:00.545422 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:00.547550 (+ 2128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:00.547598 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:00.548830 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:00.548839 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:00.549169 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:00.550145 (+ 976us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:00.550171 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:00.552797 (+ 2626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:00.556952 (+ 4155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:00.556967 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:00.556984 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:00.557513 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:00.558208 (+ 695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:00.558212 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:00.558215 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:00.558528 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:00.559268 (+ 740us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:00.559278 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:00.559692 (+ 414us) client_negotiation.cc:769] Sending connection context
0807 10:50:00.560046 (+ 354us) client_negotiation.cc:240] Negotiation successful
0807 10:50:00.561068 (+ 1022us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":377,"threads_started":1}
I20260807 10:50:00.565397 7805 heartbeater.cc:347] Connected to a master server at 127.3.73.254:43841
I20260807 10:50:00.565786 7805 heartbeater.cc:464] Registering TS with master...
I20260807 10:50:00.566778 7805 heartbeater.cc:511] Master 127.3.73.254:43841 requested a full tablet report, sending...
I20260807 10:50:00.569281 7121 ts_manager.cc:194] Registered new tserver with Master: 7bc6a60625ea42228ffa6e523fb12fd5 (127.3.73.195:40667)
I20260807 10:50:00.570243 7121 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.195@KRBTEST.COM'} at 127.3.73.195:48393
I20260807 10:50:00.575966 3367 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 10:50:00.629760 7807 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:50:00.626859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32902 (local address 127.3.73.254:43841)
0807 10:50:00.627141 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:00.627151 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:00.627541 (+ 390us) server_negotiation.cc:407] Connection header received
0807 10:50:00.627785 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:00.627790 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:00.627858 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:00.627988 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:00.629489 (+ 1501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32902: BlockingRecv error: recv got EOF from 127.0.0.1:32902 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
I20260807 10:50:00.633833 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 7220
W20260807 10:50:00.636818 3367 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 7220: Not authorized: Client connection negotiation failed: client connection to 127.3.73.193:44167: server requires authentication, but client does not have Kerberos credentials available
I20260807 10:50:00.637050 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 7220
I20260807 10:50:00.663262 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 7422
W20260807 10:50:00.666579 3367 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 7422: Not authorized: Client connection negotiation failed: client connection to 127.3.73.194:37945: server requires authentication, but client does not have Kerberos credentials available
I20260807 10:50:00.666771 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 7422
I20260807 10:50:00.702036 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 7620
W20260807 10:50:00.705516 3367 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 7620: Not authorized: Client connection negotiation failed: client connection to 127.3.73.195:40667: server requires authentication, but client does not have Kerberos credentials available
I20260807 10:50:00.705688 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 7620
I20260807 10:50:00.731631 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 7074
I20260807 10:50:00.733237 7175 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:00.842269 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 7074
2026-08-07T10:50:00Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (4521 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:50:00 dist-test-slave-x5fp krb5kdc[7831](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:50:00 dist-test-slave-x5fp krb5kdc[7831](info): set up 2 sockets
Aug 07 10:50:00 dist-test-slave-x5fp krb5kdc[7831](info): commencing operation
krb5kdc: starting...
W20260807 10:50:02.203917 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.306s user 0.002s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:50:02 dist-test-slave-x5fp krb5kdc[7831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:50:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:50:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.73.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
I20260807 10:50:02.495765 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:36249
--webserver_interface=127.3.73.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:43663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:36249
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:50:02.865693 7852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:02.866025 7852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:02.866214 7852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:02.874028 7852 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:50:02.874163 7852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:02.874259 7852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:02.874343 7852 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:50:02.874416 7852 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:50:02.885243 7852 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:43663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:36249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:36249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:02.886830 7852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:02.888362 7852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:02.902319 7862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:02.902360 7859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:02.902423 7865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:02.903156 7852 server_base.cc:1034] running on GCE node
I20260807 10:50:02.904198 7852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:02.906236 7852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:02.907549 7852 hybrid_clock.cc:648] HybridClock initialized: now 1786099802907490 us; error 57 us; skew 500 ppm
I20260807 10:50:02.907976 7852 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:02 dist-test-slave-x5fp krb5kdc[7831](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099802, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:02.911760 7852 init.cc:377] Logged in from keytab as kudu/127.3.73.254@KRBTEST.COM (short username kudu)
I20260807 10:50:02.913686 7852 webserver.cc:460] Webserver started at http://127.3.73.254:37701/ using document root <none> and password file <none>
I20260807 10:50:02.914518 7852 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:02.914705 7852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:02.915091 7852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:02.918486 7852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "6831686b76a1494eab53dc75b29bbeb6"
format_stamp: "Formatted at 2026-08-07 10:50:02 on dist-test-slave-x5fp"
server_key: "ae00e6f27ff11a76012088a7f28846d8"
server_key_iv: "4f142b0046b1c16b31f861da5b6ab041"
server_key_version: "encryptionkey@0"
I20260807 10:50:02.919368 7852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "6831686b76a1494eab53dc75b29bbeb6"
format_stamp: "Formatted at 2026-08-07 10:50:02 on dist-test-slave-x5fp"
server_key: "ae00e6f27ff11a76012088a7f28846d8"
server_key_iv: "4f142b0046b1c16b31f861da5b6ab041"
server_key_version: "encryptionkey@0"
I20260807 10:50:02.925199 7852 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 10:50:02.929350 7877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:02.930999 7852 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 10:50:02.931241 7852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "6831686b76a1494eab53dc75b29bbeb6"
format_stamp: "Formatted at 2026-08-07 10:50:02 on dist-test-slave-x5fp"
server_key: "ae00e6f27ff11a76012088a7f28846d8"
server_key_iv: "4f142b0046b1c16b31f861da5b6ab041"
server_key_version: "encryptionkey@0"
I20260807 10:50:02.931490 7852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:02.948493 7852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:02.954136 7852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:02.954542 7852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:02.976270 7852 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:36249
I20260807 10:50:02.976322 7951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:36249 every 8 connection(s)
I20260807 10:50:02.977977 7852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:50:02.982168 7952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:02.985662 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 7852
I20260807 10:50:02.985922 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:50:02.986680 3367 external_mini_cluster.cc:1428] Setting key 842accd855db305c2b0aa28dd8a26cf2
I20260807 10:50:02.995822 7952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6831686b76a1494eab53dc75b29bbeb6: Bootstrap starting.
I20260807 10:50:02.999257 7952 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6831686b76a1494eab53dc75b29bbeb6: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:03.000586 7952 log.cc:824] T 00000000000000000000000000000000 P 6831686b76a1494eab53dc75b29bbeb6: Log is configured to *not* fsync() on all Append() calls
Aug 07 10:50:02 dist-test-slave-x5fp krb5kdc[7831](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099802, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:50:03.003985 7952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6831686b76a1494eab53dc75b29bbeb6: No bootstrap required, opened a new log
I20260807 10:50:03.012859 7952 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6831686b76a1494eab53dc75b29bbeb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6831686b76a1494eab53dc75b29bbeb6" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 36249 } }
I20260807 10:50:03.013281 7952 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6831686b76a1494eab53dc75b29bbeb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:03.013459 7952 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6831686b76a1494eab53dc75b29bbeb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6831686b76a1494eab53dc75b29bbeb6, State: Initialized, Role: FOLLOWER
I20260807 10:50:03.014104 7952 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6831686b76a1494eab53dc75b29bbeb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6831686b76a1494eab53dc75b29bbeb6" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 36249 } }
I20260807 10:50:03.014407 7957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:02.988865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39596 (local address 127.3.73.254:36249)
0807 10:50:02.989859 (+ 994us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:02.989881 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:02.989948 (+ 67us) server_negotiation.cc:407] Connection header received
0807 10:50:02.991996 (+ 2048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:02.992027 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:02.992600 (+ 573us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:02.993245 (+ 645us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:02.995237 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:02.997054 (+ 1817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:02.998682 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:02.999583 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:02.999873 (+ 290us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:03.005190 (+ 5317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:03.005211 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:03.005234 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:03.005298 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:03.009165 (+ 3867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:03.010069 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:03.010088 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:03.010095 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:03.010291 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:03.010751 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:03.010765 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:03.010775 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:03.011431 (+ 656us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:03.011802 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:03.012264 (+ 462us) server_negotiation.cc:299] Negotiation successful
0807 10:50:03.012847 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":691,"thread_start_us":435,"threads_started":1}
I20260807 10:50:03.014431 7952 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6831686b76a1494eab53dc75b29bbeb6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:50:03.014623 7952 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6831686b76a1494eab53dc75b29bbeb6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:50:03.014842 7952 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6831686b76a1494eab53dc75b29bbeb6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:03.017640 7952 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6831686b76a1494eab53dc75b29bbeb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6831686b76a1494eab53dc75b29bbeb6" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 36249 } }
I20260807 10:50:03.018249 7952 leader_election.cc:302] T 00000000000000000000000000000000 P 6831686b76a1494eab53dc75b29bbeb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6831686b76a1494eab53dc75b29bbeb6; no voters:
I20260807 10:50:03.018976 3367 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 10:50:03.019059 7952 leader_election.cc:288] T 00000000000000000000000000000000 P 6831686b76a1494eab53dc75b29bbeb6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:50:03.019165 7959 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6831686b76a1494eab53dc75b29bbeb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:50:03.021899 7952 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6831686b76a1494eab53dc75b29bbeb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:50:03.025877 7959 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6831686b76a1494eab53dc75b29bbeb6 [term 1 LEADER]: Becoming Leader. State: Replica: 6831686b76a1494eab53dc75b29bbeb6, State: Running, Role: LEADER
I20260807 10:50:03.026690 7959 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6831686b76a1494eab53dc75b29bbeb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6831686b76a1494eab53dc75b29bbeb6" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 36249 } }
I20260807 10:50:03.033382 7961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6831686b76a1494eab53dc75b29bbeb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6831686b76a1494eab53dc75b29bbeb6. Latest consensus state: current_term: 1 leader_uuid: "6831686b76a1494eab53dc75b29bbeb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6831686b76a1494eab53dc75b29bbeb6" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 36249 } } }
I20260807 10:50:03.033411 7960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6831686b76a1494eab53dc75b29bbeb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6831686b76a1494eab53dc75b29bbeb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6831686b76a1494eab53dc75b29bbeb6" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 36249 } } }
I20260807 10:50:03.033888 7961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6831686b76a1494eab53dc75b29bbeb6 [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:03.033926 7960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6831686b76a1494eab53dc75b29bbeb6 [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:03.042958 7980 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:50:03.049154 7980 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:50:03.059924 7980 catalog_manager.cc:1347] Generated new cluster ID: 130264c91d934c7aa5fb4220895cce1f
I20260807 10:50:03.060063 7980 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:50:03.069048 7980 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:50:03.070637 7980 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:50:03.084429 7980 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6831686b76a1494eab53dc75b29bbeb6: Generated new TSK 0
I20260807 10:50:03.085664 7980 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 10:50:03.092056 7957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:03.077739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39604 (local address 127.3.73.254:36249)
0807 10:50:03.077940 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:03.077946 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:03.082206 (+ 4260us) server_negotiation.cc:407] Connection header received
0807 10:50:03.082549 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:03.082556 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:03.082663 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:03.082796 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:03.084353 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:03.085549 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:03.087672 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:03.088348 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:03.088498 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:03.089933 (+ 1435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:03.089941 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:03.089946 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:50:03.091073 (+ 1127us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:03.091174 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:50:03.091214 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:03.091351 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:03.091561 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 10:50:03.091672 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 10:50:03.101955 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 7852
I20260807 10:50:03.104331 7946 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:03.198863 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 7852
2026-08-07T10:50:03Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2349 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:50:03 dist-test-slave-x5fp krb5kdc[8005](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:50:03 dist-test-slave-x5fp krb5kdc[8005](info): set up 2 sockets
Aug 07 10:50:03 dist-test-slave-x5fp krb5kdc[8005](info): commencing operation
krb5kdc: starting...
W20260807 10:50:04.583877 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.337s user 0.001s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:50:04 dist-test-slave-x5fp krb5kdc[8005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099804, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:50:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:50:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.73.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
I20260807 10:50:04.864532 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:38493
--webserver_interface=127.3.73.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:35781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:38493
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:50:05.246269 8026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:05.246673 8026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:05.246780 8026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:05.254118 8026 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:50:05.254236 8026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:05.254285 8026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:05.254344 8026 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:50:05.254390 8026 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:50:05.264340 8026 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:38493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:38493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:05.265681 8026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:05.267030 8026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:05.279222 8034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:05.279399 8037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:05.279511 8035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:05.280176 8026 server_base.cc:1034] running on GCE node
I20260807 10:50:05.281123 8026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:05.282577 8026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:05.283813 8026 hybrid_clock.cc:648] HybridClock initialized: now 1786099805283750 us; error 30 us; skew 500 ppm
I20260807 10:50:05.284143 8026 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:05 dist-test-slave-x5fp krb5kdc[8005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099805, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:05.287498 8026 init.cc:377] Logged in from keytab as kudu/127.3.73.254@KRBTEST.COM (short username kudu)
I20260807 10:50:05.289186 8026 webserver.cc:460] Webserver started at http://127.3.73.254:44379/ using document root <none> and password file <none>
I20260807 10:50:05.289919 8026 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:05.290067 8026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:05.290421 8026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:05.293634 8026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "c4380d82ccd94bb5960fd8b29bf632ba"
format_stamp: "Formatted at 2026-08-07 10:50:05 on dist-test-slave-x5fp"
server_key: "b938416a1dea5fbb4b6fb0dbbe63d69b"
server_key_iv: "45ab03957ad03aca4daff2a8011f7aae"
server_key_version: "encryptionkey@0"
I20260807 10:50:05.294463 8026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "c4380d82ccd94bb5960fd8b29bf632ba"
format_stamp: "Formatted at 2026-08-07 10:50:05 on dist-test-slave-x5fp"
server_key: "b938416a1dea5fbb4b6fb0dbbe63d69b"
server_key_iv: "45ab03957ad03aca4daff2a8011f7aae"
server_key_version: "encryptionkey@0"
I20260807 10:50:05.299952 8026 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 10:50:05.303596 8051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:05.305019 8026 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 10:50:05.305209 8026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "c4380d82ccd94bb5960fd8b29bf632ba"
format_stamp: "Formatted at 2026-08-07 10:50:05 on dist-test-slave-x5fp"
server_key: "b938416a1dea5fbb4b6fb0dbbe63d69b"
server_key_iv: "45ab03957ad03aca4daff2a8011f7aae"
server_key_version: "encryptionkey@0"
I20260807 10:50:05.305429 8026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:05.327843 8026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:05.332795 8026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:05.333129 8026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:05.354779 8026 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:38493
I20260807 10:50:05.354818 8129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:38493 every 8 connection(s)
I20260807 10:50:05.356693 8026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:50:05.360869 8131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:05.363179 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 8026
I20260807 10:50:05.363445 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:50:05.364147 3367 external_mini_cluster.cc:1428] Setting key 93126b4037c0759161459af19449fcb1
I20260807 10:50:05.374787 8131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4380d82ccd94bb5960fd8b29bf632ba: Bootstrap starting.
I20260807 10:50:05.379083 8131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4380d82ccd94bb5960fd8b29bf632ba: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:05.380615 8131 log.cc:824] T 00000000000000000000000000000000 P c4380d82ccd94bb5960fd8b29bf632ba: Log is configured to *not* fsync() on all Append() calls
Aug 07 10:50:05 dist-test-slave-x5fp krb5kdc[8005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099804, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:50:05.384662 8131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4380d82ccd94bb5960fd8b29bf632ba: No bootstrap required, opened a new log
I20260807 10:50:05.391654 8135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:05.366609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56064 (local address 127.3.73.254:38493)
0807 10:50:05.367505 (+ 896us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:05.367519 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:05.367565 (+ 46us) server_negotiation.cc:407] Connection header received
0807 10:50:05.369560 (+ 1995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:05.369587 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:05.370020 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:05.370387 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:05.372452 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:05.374044 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:05.375898 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:05.376841 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:05.377031 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:05.382845 (+ 5814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:05.382857 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:05.382876 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:05.382922 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:05.386602 (+ 3680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:05.387501 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:05.387518 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:05.387526 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:05.387826 (+ 300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:05.388298 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:05.388305 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:05.388308 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:05.389004 (+ 696us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:05.389406 (+ 402us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:05.389929 (+ 523us) server_negotiation.cc:299] Negotiation successful
0807 10:50:05.390473 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":429,"threads_started":1}
I20260807 10:50:05.393951 8131 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4380d82ccd94bb5960fd8b29bf632ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4380d82ccd94bb5960fd8b29bf632ba" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 38493 } }
I20260807 10:50:05.394421 8131 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4380d82ccd94bb5960fd8b29bf632ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:05.394538 8131 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4380d82ccd94bb5960fd8b29bf632ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4380d82ccd94bb5960fd8b29bf632ba, State: Initialized, Role: FOLLOWER
I20260807 10:50:05.395262 8131 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4380d82ccd94bb5960fd8b29bf632ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4380d82ccd94bb5960fd8b29bf632ba" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 38493 } }
I20260807 10:50:05.395579 8131 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4380d82ccd94bb5960fd8b29bf632ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:50:05.395730 8131 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4380d82ccd94bb5960fd8b29bf632ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:50:05.395886 8131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4380d82ccd94bb5960fd8b29bf632ba [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:05.398964 8131 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4380d82ccd94bb5960fd8b29bf632ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4380d82ccd94bb5960fd8b29bf632ba" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 38493 } }
I20260807 10:50:05.399628 8131 leader_election.cc:302] T 00000000000000000000000000000000 P c4380d82ccd94bb5960fd8b29bf632ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4380d82ccd94bb5960fd8b29bf632ba; no voters:
I20260807 10:50:05.400241 8131 leader_election.cc:288] T 00000000000000000000000000000000 P c4380d82ccd94bb5960fd8b29bf632ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:50:05.400445 8137 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4380d82ccd94bb5960fd8b29bf632ba [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:50:05.401392 8137 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4380d82ccd94bb5960fd8b29bf632ba [term 1 LEADER]: Becoming Leader. State: Replica: c4380d82ccd94bb5960fd8b29bf632ba, State: Running, Role: LEADER
I20260807 10:50:05.402117 8137 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4380d82ccd94bb5960fd8b29bf632ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4380d82ccd94bb5960fd8b29bf632ba" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 38493 } }
I20260807 10:50:05.402957 8131 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4380d82ccd94bb5960fd8b29bf632ba [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:50:05.405557 8139 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4380d82ccd94bb5960fd8b29bf632ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4380d82ccd94bb5960fd8b29bf632ba. Latest consensus state: current_term: 1 leader_uuid: "c4380d82ccd94bb5960fd8b29bf632ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4380d82ccd94bb5960fd8b29bf632ba" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 38493 } } }
I20260807 10:50:05.405591 8138 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4380d82ccd94bb5960fd8b29bf632ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4380d82ccd94bb5960fd8b29bf632ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4380d82ccd94bb5960fd8b29bf632ba" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 38493 } } }
I20260807 10:50:05.406117 8138 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4380d82ccd94bb5960fd8b29bf632ba [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:05.406113 8139 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4380d82ccd94bb5960fd8b29bf632ba [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:05.413956 8150 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:50:05.419512 8150 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:50:05.429412 8150 catalog_manager.cc:1347] Generated new cluster ID: 2bf1480dd73e4be890369cfcc6723a6b
I20260807 10:50:05.429557 8150 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:50:05.454185 8150 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:50:05.455963 8150 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:50:05.463259 8150 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4380d82ccd94bb5960fd8b29bf632ba: Generated new TSK 0
I20260807 10:50:05.464357 8150 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.73.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab.
Entry for principal kudu/127.3.73.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab.
I20260807 10:50:05.537254 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.193:0
--local_ip_for_outbound_sockets=127.3.73.193
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:38493
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:35781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:50:05.894961 8167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:05.895257 8167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:05.895439 8167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:05.903167 8167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:05.903349 8167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:05.903551 8167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.193
I20260807 10:50:05.914594 8167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:38493
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.193
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:05.916013 8167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:05.917557 8167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:05.931859 8178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:05.931861 8175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:05.931908 8173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:05.932833 8167 server_base.cc:1034] running on GCE node
I20260807 10:50:05.933535 8167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:05.934645 8167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:05.935891 8167 hybrid_clock.cc:648] HybridClock initialized: now 1786099805935838 us; error 70 us; skew 500 ppm
I20260807 10:50:05.936245 8167 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:05 dist-test-slave-x5fp krb5kdc[8005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099805, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:05.939673 8167 init.cc:377] Logged in from keytab as kudu/127.3.73.193@KRBTEST.COM (short username kudu)
I20260807 10:50:05.941383 8167 webserver.cc:460] Webserver started at http://127.3.73.193:38659/ using document root <none> and password file <none>
I20260807 10:50:05.942101 8167 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:05.942210 8167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:05.942478 8167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:05.945689 8167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-0/data/instance:
uuid: "e46dd8814bca4d42af0e8bc90b429a0b"
format_stamp: "Formatted at 2026-08-07 10:50:05 on dist-test-slave-x5fp"
server_key: "d41d28eae0600b45bad55c9b5b6c45f1"
server_key_iv: "2e5c0911a15300af8a5b7ed17b462768"
server_key_version: "encryptionkey@0"
I20260807 10:50:05.946597 8167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance:
uuid: "e46dd8814bca4d42af0e8bc90b429a0b"
format_stamp: "Formatted at 2026-08-07 10:50:05 on dist-test-slave-x5fp"
server_key: "d41d28eae0600b45bad55c9b5b6c45f1"
server_key_iv: "2e5c0911a15300af8a5b7ed17b462768"
server_key_version: "encryptionkey@0"
I20260807 10:50:05.952119 8167 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260807 10:50:05.956508 8192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:05.958498 8167 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 10:50:05.958702 8167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-0/wal
uuid: "e46dd8814bca4d42af0e8bc90b429a0b"
format_stamp: "Formatted at 2026-08-07 10:50:05 on dist-test-slave-x5fp"
server_key: "d41d28eae0600b45bad55c9b5b6c45f1"
server_key_iv: "2e5c0911a15300af8a5b7ed17b462768"
server_key_version: "encryptionkey@0"
I20260807 10:50:05.958986 8167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:05.973093 8167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:05.978029 8167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:05.978349 8167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:05.979384 8167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:50:05.980983 8167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:50:05.981072 8167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:05.981170 8167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:50:05.981217 8167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:06.019814 8167 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.193:46387
I20260807 10:50:06.019932 8363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.193:46387 every 8 connection(s)
I20260807 10:50:06.021240 8167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
I20260807 10:50:06.025465 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 8167
I20260807 10:50:06.025700 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance
I20260807 10:50:06.026530 3367 external_mini_cluster.cc:1428] Setting key fe3702c0ca4a216f90ff76b171466fdb
Aug 07 10:50:06 dist-test-slave-x5fp krb5kdc[8005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099805, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:50:06.045439 8366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:06.024301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:59319 (local address 127.3.73.254:38493)
0807 10:50:06.024963 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:06.024968 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:06.026057 (+ 1089us) server_negotiation.cc:407] Connection header received
0807 10:50:06.028199 (+ 2142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:06.028205 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:06.028295 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:06.028439 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:06.030147 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:06.030913 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:06.032613 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:06.033120 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:06.033273 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:06.037613 (+ 4340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:06.037620 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:06.037624 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:06.037663 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:06.041248 (+ 3585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:06.042197 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:06.042207 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:06.042214 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:06.042399 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:06.042947 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:06.042956 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:06.042963 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:06.043408 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:06.043586 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:06.044472 (+ 886us) server_negotiation.cc:299] Negotiation successful
0807 10:50:06.044865 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":297,"threads_started":1}
I20260807 10:50:06.046118 8367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:06.024860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:38493 (local address 127.3.73.193:59319)
0807 10:50:06.025862 (+ 1002us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:06.025918 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:06.027907 (+ 1989us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:06.028698 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:06.028707 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:06.029351 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:06.029960 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:06.029978 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:06.031114 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:06.031118 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:06.032437 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:06.032457 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:06.033397 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:06.033405 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:06.033653 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:06.034478 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:06.034503 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:06.037318 (+ 2815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:06.041457 (+ 4139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:06.041467 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:06.041490 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:06.042025 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:06.042551 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:06.042555 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:06.042558 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:06.042786 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:06.043578 (+ 792us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:06.043586 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:06.044015 (+ 429us) client_negotiation.cc:769] Sending connection context
0807 10:50:06.044281 (+ 266us) client_negotiation.cc:240] Negotiation successful
0807 10:50:06.044710 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":378,"threads_started":1}
I20260807 10:50:06.048060 8364 heartbeater.cc:347] Connected to a master server at 127.3.73.254:38493
I20260807 10:50:06.048518 8364 heartbeater.cc:464] Registering TS with master...
I20260807 10:50:06.049630 8364 heartbeater.cc:511] Master 127.3.73.254:38493 requested a full tablet report, sending...
I20260807 10:50:06.052506 8080 ts_manager.cc:194] Registered new tserver with Master: e46dd8814bca4d42af0e8bc90b429a0b (127.3.73.193:46387)
I20260807 10:50:06.054132 8080 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.193@KRBTEST.COM'} at 127.3.73.193:59319
WARNING: no policy specified for kudu/127.3.73.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab.
Entry for principal kudu/127.3.73.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab.
I20260807 10:50:06.094573 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.194:0
--local_ip_for_outbound_sockets=127.3.73.194
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:38493
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:35781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:50:06.445681 8370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:06.445990 8370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:06.446159 8370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:06.453509 8370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:06.453629 8370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:06.453747 8370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.194
I20260807 10:50:06.463903 8370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:38493
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.194
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:06.465519 8370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:06.466835 8370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:06.480661 8376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:06.480732 8379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:06.480775 8377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:06.481822 8370 server_base.cc:1034] running on GCE node
I20260807 10:50:06.482548 8370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:06.483629 8370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:06.484870 8370 hybrid_clock.cc:648] HybridClock initialized: now 1786099806484788 us; error 42 us; skew 500 ppm
I20260807 10:50:06.485225 8370 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:06 dist-test-slave-x5fp krb5kdc[8005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099806, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:06.488821 8370 init.cc:377] Logged in from keytab as kudu/127.3.73.194@KRBTEST.COM (short username kudu)
I20260807 10:50:06.490478 8370 webserver.cc:460] Webserver started at http://127.3.73.194:34627/ using document root <none> and password file <none>
I20260807 10:50:06.491305 8370 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:06.491464 8370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:06.491791 8370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:06.495698 8370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-1/data/instance:
uuid: "409dafa33a094866be33fdc13bfc46cf"
format_stamp: "Formatted at 2026-08-07 10:50:06 on dist-test-slave-x5fp"
server_key: "60f58cb3704cb3bfcd3b9c5574a703d2"
server_key_iv: "3e8ba5ad2927edcb407db8fc63156134"
server_key_version: "encryptionkey@0"
I20260807 10:50:06.496929 8370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance:
uuid: "409dafa33a094866be33fdc13bfc46cf"
format_stamp: "Formatted at 2026-08-07 10:50:06 on dist-test-slave-x5fp"
server_key: "60f58cb3704cb3bfcd3b9c5574a703d2"
server_key_iv: "3e8ba5ad2927edcb407db8fc63156134"
server_key_version: "encryptionkey@0"
I20260807 10:50:06.503219 8370 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260807 10:50:06.507571 8389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:06.509217 8370 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 10:50:06.509434 8370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-1/wal
uuid: "409dafa33a094866be33fdc13bfc46cf"
format_stamp: "Formatted at 2026-08-07 10:50:06 on dist-test-slave-x5fp"
server_key: "60f58cb3704cb3bfcd3b9c5574a703d2"
server_key_iv: "3e8ba5ad2927edcb407db8fc63156134"
server_key_version: "encryptionkey@0"
I20260807 10:50:06.509673 8370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:06.526227 8370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:06.531482 8370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:06.531877 8370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:06.533098 8370 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:50:06.534682 8370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:50:06.534771 8370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:06.534935 8370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:50:06.534998 8370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:06.571741 8370 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.194:36709
I20260807 10:50:06.571784 8552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.194:36709 every 8 connection(s)
I20260807 10:50:06.573299 8370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
I20260807 10:50:06.583432 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 8370
I20260807 10:50:06.583639 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance
I20260807 10:50:06.584224 3367 external_mini_cluster.cc:1428] Setting key 4adfa6995a669995e711b67f5e8d29f8
Aug 07 10:50:06 dist-test-slave-x5fp krb5kdc[8005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099806, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:50:06.597344 8555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:06.576318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:43633 (local address 127.3.73.254:38493)
0807 10:50:06.576890 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:06.576896 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:06.578018 (+ 1122us) server_negotiation.cc:407] Connection header received
0807 10:50:06.579988 (+ 1970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:06.579994 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:06.580101 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:06.580244 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:06.581983 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:06.582713 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:06.585180 (+ 2467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:06.585668 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:06.585880 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:06.590129 (+ 4249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:06.590135 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:06.590138 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:06.590172 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:06.593333 (+ 3161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:06.594330 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:06.594337 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:06.594340 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:06.594505 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:06.595088 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:06.595100 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:06.595103 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:06.595439 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:06.595637 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:06.596648 (+ 1011us) server_negotiation.cc:299] Negotiation successful
0807 10:50:06.596957 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":369,"thread_start_us":244,"threads_started":1}
I20260807 10:50:06.598532 8556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:06.577011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:38493 (local address 127.3.73.194:43633)
0807 10:50:06.577828 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:06.577870 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:06.579652 (+ 1782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:06.580510 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:06.580519 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:06.581166 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:06.581785 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:06.581803 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:06.582891 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:06.582897 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:06.584976 (+ 2079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:06.584990 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:06.585989 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:06.585997 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:06.586342 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:06.587187 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:06.587212 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:06.589852 (+ 2640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:06.593542 (+ 3690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:06.593555 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:06.593579 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:06.594132 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:06.594672 (+ 540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:06.594679 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:06.594685 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:06.594925 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:06.595605 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:06.595615 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:06.596200 (+ 585us) client_negotiation.cc:769] Sending connection context
0807 10:50:06.596526 (+ 326us) client_negotiation.cc:240] Negotiation successful
0807 10:50:06.597145 (+ 619us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":343,"threads_started":1}
I20260807 10:50:06.600179 8553 heartbeater.cc:347] Connected to a master server at 127.3.73.254:38493
I20260807 10:50:06.600558 8553 heartbeater.cc:464] Registering TS with master...
I20260807 10:50:06.601617 8553 heartbeater.cc:511] Master 127.3.73.254:38493 requested a full tablet report, sending...
I20260807 10:50:06.603384 8080 ts_manager.cc:194] Registered new tserver with Master: 409dafa33a094866be33fdc13bfc46cf (127.3.73.194:36709)
I20260807 10:50:06.604310 8080 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.194@KRBTEST.COM'} at 127.3.73.194:43633
WARNING: no policy specified for kudu/127.3.73.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab.
Entry for principal kudu/127.3.73.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab.
I20260807 10:50:06.646438 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.195:0
--local_ip_for_outbound_sockets=127.3.73.195
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:38493
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:35781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:50:07.009508 8559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:07.009833 8559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:07.009934 8559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:07.017616 8559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:07.017791 8559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:07.017913 8559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.195
I20260807 10:50:07.028218 8559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:38493
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.195
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:07.029564 8559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:07.030862 8559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:07.045004 8567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:07.045009 8566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:07.045221 8569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:07.045835 8559 server_base.cc:1034] running on GCE node
I20260807 10:50:07.046644 8559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:07.047737 8559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:07.049026 8559 hybrid_clock.cc:648] HybridClock initialized: now 1786099807048974 us; error 40 us; skew 500 ppm
I20260807 10:50:07.049389 8559 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:07 dist-test-slave-x5fp krb5kdc[8005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099807, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:07.052959 8559 init.cc:377] Logged in from keytab as kudu/127.3.73.195@KRBTEST.COM (short username kudu)
I20260807 10:50:07.054550 8559 webserver.cc:460] Webserver started at http://127.3.73.195:44125/ using document root <none> and password file <none>
I20260807 10:50:07.055351 8559 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:07.055516 8559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:07.055873 8559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:07.058272 8364 heartbeater.cc:503] Master 127.3.73.254:38493 was elected leader, sending a full tablet report...
I20260807 10:50:07.059275 8559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-2/data/instance:
uuid: "39978f46e6bd4a1388a448ff8e9b1308"
format_stamp: "Formatted at 2026-08-07 10:50:07 on dist-test-slave-x5fp"
server_key: "c95f3405d0dc3fdc9b715099d4b7c191"
server_key_iv: "1746dd890776159d9d7ab73432dc331b"
server_key_version: "encryptionkey@0"
I20260807 10:50:07.060223 8559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance:
uuid: "39978f46e6bd4a1388a448ff8e9b1308"
format_stamp: "Formatted at 2026-08-07 10:50:07 on dist-test-slave-x5fp"
server_key: "c95f3405d0dc3fdc9b715099d4b7c191"
server_key_iv: "1746dd890776159d9d7ab73432dc331b"
server_key_version: "encryptionkey@0"
I20260807 10:50:07.066020 8559 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 10:50:07.069897 8579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:07.071520 8559 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 10:50:07.071762 8559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-2/wal
uuid: "39978f46e6bd4a1388a448ff8e9b1308"
format_stamp: "Formatted at 2026-08-07 10:50:07 on dist-test-slave-x5fp"
server_key: "c95f3405d0dc3fdc9b715099d4b7c191"
server_key_iv: "1746dd890776159d9d7ab73432dc331b"
server_key_version: "encryptionkey@0"
I20260807 10:50:07.072021 8559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:07.092957 8559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:07.098255 8559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:07.098634 8559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:07.099819 8559 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:50:07.103621 8559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:50:07.103749 8559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:07.103931 8559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:50:07.104019 8559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:07.142817 8559 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.195:37667
I20260807 10:50:07.142868 8748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.195:37667 every 8 connection(s)
I20260807 10:50:07.144438 8559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
I20260807 10:50:07.154671 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 8559
I20260807 10:50:07.154964 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance
I20260807 10:50:07.155473 3367 external_mini_cluster.cc:1428] Setting key e3751e2ffaf615f6b15b7ab3fe9debbb
Aug 07 10:50:07 dist-test-slave-x5fp krb5kdc[8005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099807, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:50:07.168905 8752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:07.147592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:54729 (local address 127.3.73.254:38493)
0807 10:50:07.148137 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:07.148142 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:07.149216 (+ 1074us) server_negotiation.cc:407] Connection header received
0807 10:50:07.151127 (+ 1911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:07.151133 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:07.151240 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:07.151381 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:07.153107 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:07.154022 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:07.155921 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:07.156442 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:07.156582 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:07.161393 (+ 4811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:07.161399 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:07.161402 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:07.161461 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:07.164562 (+ 3101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:07.165728 (+ 1166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:07.165733 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:07.165735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:07.165901 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:07.166822 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:07.166827 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:07.166830 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:07.167172 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:07.167330 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:07.168246 (+ 916us) server_negotiation.cc:299] Negotiation successful
0807 10:50:07.168475 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":295,"threads_started":1}
I20260807 10:50:07.169894 8755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:07.148051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:38493 (local address 127.3.73.195:54729)
0807 10:50:07.149021 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:07.149065 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:07.150816 (+ 1751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:07.151649 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:07.151659 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:07.152245 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:07.152898 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:07.152918 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:07.154213 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:07.154222 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:07.155748 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:07.155763 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:07.156786 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:07.156796 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:07.157190 (+ 394us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:07.158083 (+ 893us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:07.158118 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:07.161122 (+ 3004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:07.164899 (+ 3777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:07.164908 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:07.164924 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:07.165524 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:07.166139 (+ 615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:07.166143 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:07.166146 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:07.166510 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:07.167332 (+ 822us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:07.167341 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:07.167751 (+ 410us) client_negotiation.cc:769] Sending connection context
0807 10:50:07.168063 (+ 312us) client_negotiation.cc:240] Negotiation successful
0807 10:50:07.168534 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":340,"threads_started":1}
I20260807 10:50:07.171607 8749 heartbeater.cc:347] Connected to a master server at 127.3.73.254:38493
I20260807 10:50:07.171980 8749 heartbeater.cc:464] Registering TS with master...
I20260807 10:50:07.172904 8749 heartbeater.cc:511] Master 127.3.73.254:38493 requested a full tablet report, sending...
I20260807 10:50:07.174548 8079 ts_manager.cc:194] Registered new tserver with Master: 39978f46e6bd4a1388a448ff8e9b1308 (127.3.73.195:37667)
I20260807 10:50:07.175621 8079 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.195@KRBTEST.COM'} at 127.3.73.195:54729
I20260807 10:50:07.185446 3367 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 10:50:07 dist-test-slave-x5fp krb5kdc[8005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099807, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 10:50:07 dist-test-slave-x5fp krb5kdc[8005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099807, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:50:07.255666 8752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:07.239288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56080 (local address 127.3.73.254:38493)
0807 10:50:07.239516 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:07.239521 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:07.239993 (+ 472us) server_negotiation.cc:407] Connection header received
0807 10:50:07.240165 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:07.240170 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:07.240244 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:07.240351 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:07.242121 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:07.242975 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:07.244491 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:07.245099 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:07.245214 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:07.249718 (+ 4504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:07.249723 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:07.249726 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:07.249770 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:07.252486 (+ 2716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:07.253343 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:07.253348 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:07.253351 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:07.253502 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:07.253908 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:07.253913 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:07.253915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:07.254318 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:07.254513 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:07.254910 (+ 397us) server_negotiation.cc:299] Negotiation successful
0807 10:50:07.255241 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260807 10:50:07.256405 8079 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:56080
I20260807 10:50:07.258366 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 8167
Aug 07 10:50:07 dist-test-slave-x5fp krb5kdc[8005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099807, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.3.73.193@KRBTEST.COM
I20260807 10:50:07.275568 8767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:07.259442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54280 (local address 127.3.73.193:46387)
0807 10:50:07.260160 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:07.260167 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:07.260195 (+ 28us) server_negotiation.cc:407] Connection header received
0807 10:50:07.260263 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:07.260272 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:07.260496 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:07.260648 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:07.261923 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:07.263069 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:07.264326 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:07.265055 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:07.265174 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:07.269447 (+ 4273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:07.269459 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:07.269475 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:07.269514 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:07.272852 (+ 3338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:07.273546 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:07.273555 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:07.273557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:07.273780 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:07.274167 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:07.274170 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:07.274172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:07.274531 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:07.274765 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:07.275051 (+ 286us) server_negotiation.cc:299] Negotiation successful
0807 10:50:07.275257 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":354,"threads_started":1}
W20260807 10:50:07.277112 8354 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:54280
W20260807 10:50:07.278329 3367 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 8167: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 10:50:07.278571 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 8167
I20260807 10:50:07.310484 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 8370
Aug 07 10:50:07 dist-test-slave-x5fp krb5kdc[8005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099807, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.3.73.194@KRBTEST.COM
I20260807 10:50:07.330430 8769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:07.311571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43318 (local address 127.3.73.194:36709)
0807 10:50:07.312128 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:07.312135 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:07.312162 (+ 27us) server_negotiation.cc:407] Connection header received
0807 10:50:07.312225 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:07.312231 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:07.312470 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:07.312607 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:07.314165 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:07.315985 (+ 1820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:07.318120 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:07.318792 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:07.318912 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:07.323812 (+ 4900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:07.323825 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:07.323842 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:07.323883 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:07.327367 (+ 3484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:07.328165 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:07.328173 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:07.328176 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:07.328322 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:07.328709 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:07.328712 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:07.328715 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:07.329190 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:07.329462 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:07.329766 (+ 304us) server_negotiation.cc:299] Negotiation successful
0807 10:50:07.330040 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":272,"threads_started":1}
W20260807 10:50:07.331804 8547 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:43318
W20260807 10:50:07.333109 3367 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 8370: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 10:50:07.333278 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 8370
I20260807 10:50:07.363924 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 8559
Aug 07 10:50:07 dist-test-slave-x5fp krb5kdc[8005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099807, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.3.73.195@KRBTEST.COM
I20260807 10:50:07.383131 8771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:07.365198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36654 (local address 127.3.73.195:37667)
0807 10:50:07.365962 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:07.365970 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:07.366001 (+ 31us) server_negotiation.cc:407] Connection header received
0807 10:50:07.366065 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:07.366071 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:07.366293 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:07.366495 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:07.367968 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:07.369190 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:07.370525 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:07.371106 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:07.371260 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:07.376177 (+ 4917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:07.376191 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:07.376207 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:07.376249 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:07.379637 (+ 3388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:07.380554 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:07.380573 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:07.380576 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:07.380795 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:07.381238 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:07.381246 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:07.381249 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:07.381654 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:07.382038 (+ 384us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:07.382384 (+ 346us) server_negotiation.cc:299] Negotiation successful
0807 10:50:07.382708 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":421,"threads_started":1}
W20260807 10:50:07.384688 8743 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36654
W20260807 10:50:07.386082 3367 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 8559: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 10:50:07.386263 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 8559
I20260807 10:50:07.411718 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 8026
I20260807 10:50:07.413173 8122 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:07.528575 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 8026
2026-08-07T10:50:07Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (4343 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:50:07 dist-test-slave-x5fp krb5kdc[8777](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:50:07 dist-test-slave-x5fp krb5kdc[8777](info): set up 2 sockets
Aug 07 10:50:07 dist-test-slave-x5fp krb5kdc[8777](info): commencing operation
krb5kdc: starting...
W20260807 10:50:08.879258 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.289s user 0.001s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:50:09 dist-test-slave-x5fp krb5kdc[8777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099809, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:50:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:50:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.73.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
I20260807 10:50:09.171264 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:32785
--webserver_interface=127.3.73.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:44743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:32785
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:50:09.546650 8804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:09.546998 8804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:09.547093 8804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:09.554962 8804 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:50:09.555114 8804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:09.555166 8804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:09.555227 8804 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:50:09.555286 8804 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:50:09.566569 8804 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:44743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:32785
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:32785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:09.568054 8804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:09.569573 8804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:09.582790 8810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:09.582840 8811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:09.583004 8815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:09.583828 8804 server_base.cc:1034] running on GCE node
I20260807 10:50:09.584898 8804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:09.586508 8804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:09.587769 8804 hybrid_clock.cc:648] HybridClock initialized: now 1786099809587694 us; error 40 us; skew 500 ppm
I20260807 10:50:09.588234 8804 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:09 dist-test-slave-x5fp krb5kdc[8777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099809, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:09.592054 8804 init.cc:377] Logged in from keytab as kudu/127.3.73.254@KRBTEST.COM (short username kudu)
I20260807 10:50:09.593942 8804 webserver.cc:460] Webserver started at http://127.3.73.254:39485/ using document root <none> and password file <none>
I20260807 10:50:09.594710 8804 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:09.594877 8804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:09.595227 8804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:09.598582 8804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "8de229aaf0b14070b05fdf2cb5f22fad"
format_stamp: "Formatted at 2026-08-07 10:50:09 on dist-test-slave-x5fp"
server_key: "8d9b182f0aa7cadd5f1c4daae28860d6"
server_key_iv: "619b3cd29955ef359640e819f40fe6dc"
server_key_version: "encryptionkey@0"
I20260807 10:50:09.599494 8804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "8de229aaf0b14070b05fdf2cb5f22fad"
format_stamp: "Formatted at 2026-08-07 10:50:09 on dist-test-slave-x5fp"
server_key: "8d9b182f0aa7cadd5f1c4daae28860d6"
server_key_iv: "619b3cd29955ef359640e819f40fe6dc"
server_key_version: "encryptionkey@0"
I20260807 10:50:09.605590 8804 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260807 10:50:09.609766 8824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:09.611366 8804 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 10:50:09.611564 8804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "8de229aaf0b14070b05fdf2cb5f22fad"
format_stamp: "Formatted at 2026-08-07 10:50:09 on dist-test-slave-x5fp"
server_key: "8d9b182f0aa7cadd5f1c4daae28860d6"
server_key_iv: "619b3cd29955ef359640e819f40fe6dc"
server_key_version: "encryptionkey@0"
I20260807 10:50:09.611797 8804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:09.627499 8804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:09.632544 8804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:09.632907 8804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:09.655397 8804 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:32785
I20260807 10:50:09.655416 8892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:32785 every 8 connection(s)
I20260807 10:50:09.657110 8804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:50:09.661314 8895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:09.662094 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 8804
I20260807 10:50:09.662376 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:50:09.663211 3367 external_mini_cluster.cc:1428] Setting key a7b13205208de0f775366780c8a24afc
I20260807 10:50:09.676918 8895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8de229aaf0b14070b05fdf2cb5f22fad: Bootstrap starting.
Aug 07 10:50:09 dist-test-slave-x5fp krb5kdc[8777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099809, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:50:09.680975 8895 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8de229aaf0b14070b05fdf2cb5f22fad: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:09.682292 8895 log.cc:824] T 00000000000000000000000000000000 P 8de229aaf0b14070b05fdf2cb5f22fad: Log is configured to *not* fsync() on all Append() calls
I20260807 10:50:09.686296 8895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8de229aaf0b14070b05fdf2cb5f22fad: No bootstrap required, opened a new log
I20260807 10:50:09.689394 8899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:09.665549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35302 (local address 127.3.73.254:32785)
0807 10:50:09.666443 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:09.666455 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:09.666501 (+ 46us) server_negotiation.cc:407] Connection header received
0807 10:50:09.668243 (+ 1742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:09.668273 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:09.668788 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:09.669184 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:09.671120 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:09.672483 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:09.674361 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:09.675128 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:09.675296 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:09.680438 (+ 5142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:09.680448 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:09.680466 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:09.680520 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:09.684351 (+ 3831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:09.685500 (+ 1149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:09.685509 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:09.685516 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:09.685716 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:09.686186 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:09.686190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:09.686192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:09.686893 (+ 701us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:09.687226 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:09.687673 (+ 447us) server_negotiation.cc:299] Negotiation successful
0807 10:50:09.688093 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":414,"threads_started":1}
I20260807 10:50:09.694883 8895 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8de229aaf0b14070b05fdf2cb5f22fad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8de229aaf0b14070b05fdf2cb5f22fad" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 32785 } }
I20260807 10:50:09.695327 8895 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8de229aaf0b14070b05fdf2cb5f22fad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:09.695446 8895 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8de229aaf0b14070b05fdf2cb5f22fad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8de229aaf0b14070b05fdf2cb5f22fad, State: Initialized, Role: FOLLOWER
I20260807 10:50:09.696028 8895 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8de229aaf0b14070b05fdf2cb5f22fad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8de229aaf0b14070b05fdf2cb5f22fad" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 32785 } }
I20260807 10:50:09.696290 8895 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8de229aaf0b14070b05fdf2cb5f22fad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:50:09.696439 8895 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8de229aaf0b14070b05fdf2cb5f22fad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:50:09.696611 8895 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8de229aaf0b14070b05fdf2cb5f22fad [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:09.699188 8895 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8de229aaf0b14070b05fdf2cb5f22fad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8de229aaf0b14070b05fdf2cb5f22fad" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 32785 } }
I20260807 10:50:09.699728 8895 leader_election.cc:302] T 00000000000000000000000000000000 P 8de229aaf0b14070b05fdf2cb5f22fad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8de229aaf0b14070b05fdf2cb5f22fad; no voters:
I20260807 10:50:09.700342 8895 leader_election.cc:288] T 00000000000000000000000000000000 P 8de229aaf0b14070b05fdf2cb5f22fad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:50:09.700495 8901 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8de229aaf0b14070b05fdf2cb5f22fad [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:50:09.701243 8901 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8de229aaf0b14070b05fdf2cb5f22fad [term 1 LEADER]: Becoming Leader. State: Replica: 8de229aaf0b14070b05fdf2cb5f22fad, State: Running, Role: LEADER
I20260807 10:50:09.701844 8901 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8de229aaf0b14070b05fdf2cb5f22fad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8de229aaf0b14070b05fdf2cb5f22fad" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 32785 } }
I20260807 10:50:09.702632 8895 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8de229aaf0b14070b05fdf2cb5f22fad [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:50:09.704603 8902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8de229aaf0b14070b05fdf2cb5f22fad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8de229aaf0b14070b05fdf2cb5f22fad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8de229aaf0b14070b05fdf2cb5f22fad" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 32785 } } }
I20260807 10:50:09.705071 8902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8de229aaf0b14070b05fdf2cb5f22fad [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:09.705811 8903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8de229aaf0b14070b05fdf2cb5f22fad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8de229aaf0b14070b05fdf2cb5f22fad. Latest consensus state: current_term: 1 leader_uuid: "8de229aaf0b14070b05fdf2cb5f22fad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8de229aaf0b14070b05fdf2cb5f22fad" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 32785 } } }
I20260807 10:50:09.706216 8903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8de229aaf0b14070b05fdf2cb5f22fad [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:09.709400 8907 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:50:09.716284 8907 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:50:09.726297 8907 catalog_manager.cc:1347] Generated new cluster ID: 6d6fb2a38e4f425b9ff707da80e86794
I20260807 10:50:09.726464 8907 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:50:09.736375 8907 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:50:09.737964 8907 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:50:09.749827 8907 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8de229aaf0b14070b05fdf2cb5f22fad: Generated new TSK 0
I20260807 10:50:09.751106 8907 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.73.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab.
Entry for principal kudu/127.3.73.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab.
I20260807 10:50:09.825843 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.193:0
--local_ip_for_outbound_sockets=127.3.73.193
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:32785
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:44743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:50:10.203732 8928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:10.204144 8928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:10.204324 8928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:10.212218 8928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:10.212388 8928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:10.212637 8928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.193
I20260807 10:50:10.223326 8928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:44743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:32785
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.193
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:10.224831 8928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:10.226302 8928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:10.240830 8933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:10.240912 8935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:10.240873 8939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:10.241966 8928 server_base.cc:1034] running on GCE node
I20260807 10:50:10.242755 8928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:10.243882 8928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:10.245134 8928 hybrid_clock.cc:648] HybridClock initialized: now 1786099810245090 us; error 35 us; skew 500 ppm
I20260807 10:50:10.245501 8928 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:10 dist-test-slave-x5fp krb5kdc[8777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099810, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:10.257069 8928 init.cc:377] Logged in from keytab as kudu/127.3.73.193@KRBTEST.COM (short username kudu)
I20260807 10:50:10.258667 8928 webserver.cc:460] Webserver started at http://127.3.73.193:39257/ using document root <none> and password file <none>
I20260807 10:50:10.259397 8928 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:10.259510 8928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:10.259802 8928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:10.262950 8928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-0/data/instance:
uuid: "ae218d25dacb45dca53d947e3ca3339c"
format_stamp: "Formatted at 2026-08-07 10:50:10 on dist-test-slave-x5fp"
server_key: "8dd1d2c5b4e6002161eb156aa705d895"
server_key_iv: "291444b15c68b621d2e957156f41413d"
server_key_version: "encryptionkey@0"
I20260807 10:50:10.263849 8928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance:
uuid: "ae218d25dacb45dca53d947e3ca3339c"
format_stamp: "Formatted at 2026-08-07 10:50:10 on dist-test-slave-x5fp"
server_key: "8dd1d2c5b4e6002161eb156aa705d895"
server_key_iv: "291444b15c68b621d2e957156f41413d"
server_key_version: "encryptionkey@0"
I20260807 10:50:10.269881 8928 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 10:50:10.273886 8953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:10.275430 8928 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 10:50:10.275590 8928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-0/wal
uuid: "ae218d25dacb45dca53d947e3ca3339c"
format_stamp: "Formatted at 2026-08-07 10:50:10 on dist-test-slave-x5fp"
server_key: "8dd1d2c5b4e6002161eb156aa705d895"
server_key_iv: "291444b15c68b621d2e957156f41413d"
server_key_version: "encryptionkey@0"
I20260807 10:50:10.275799 8928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:10.298316 8928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:10.303014 8928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:10.303275 8928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:10.304298 8928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:50:10.305709 8928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:50:10.305799 8928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:10.305887 8928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:50:10.305932 8928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:10.344236 8928 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.193:42059
I20260807 10:50:10.344300 9121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.193:42059 every 8 connection(s)
I20260807 10:50:10.345899 8928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
I20260807 10:50:10.355665 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 8928
I20260807 10:50:10.355867 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance
I20260807 10:50:10.356478 3367 external_mini_cluster.cc:1428] Setting key a7fbf8ef9ecc2a0b4bc13f408d2ff2bf
Aug 07 10:50:10 dist-test-slave-x5fp krb5kdc[8777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099810, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:50:10.370985 9125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:10.349494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:45589 (local address 127.3.73.254:32785)
0807 10:50:10.350084 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:10.350089 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:10.351143 (+ 1054us) server_negotiation.cc:407] Connection header received
0807 10:50:10.353294 (+ 2151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:10.353300 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:10.353408 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:10.353540 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:10.356032 (+ 2492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:10.356962 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:10.359102 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:10.359559 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:10.359673 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:10.364535 (+ 4862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:10.364541 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:10.364544 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:10.364584 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:10.367564 (+ 2980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:10.368435 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:10.368441 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:10.368444 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:10.368582 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:10.369080 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:10.369086 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:10.369089 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:10.369379 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:10.369544 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:10.370318 (+ 774us) server_negotiation.cc:299] Negotiation successful
0807 10:50:10.370575 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":303,"threads_started":1}
I20260807 10:50:10.372172 9126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:10.350112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:32785 (local address 127.3.73.193:45589)
0807 10:50:10.350950 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:10.350993 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:10.352986 (+ 1993us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:10.353825 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:10.353834 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:10.354425 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:10.355061 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:10.355080 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:10.357169 (+ 2089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:10.357180 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:10.358918 (+ 1738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:10.358936 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:10.359873 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:10.359883 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:10.360238 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:10.361186 (+ 948us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:10.361235 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:10.364226 (+ 2991us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:10.367757 (+ 3531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:10.367768 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:10.367787 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:10.368296 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:10.368703 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:10.368708 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:10.368711 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:10.368957 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:10.369519 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:10.369528 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:10.369910 (+ 382us) client_negotiation.cc:769] Sending connection context
0807 10:50:10.370192 (+ 282us) client_negotiation.cc:240] Negotiation successful
0807 10:50:10.370802 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":339,"threads_started":1}
I20260807 10:50:10.374001 9123 heartbeater.cc:347] Connected to a master server at 127.3.73.254:32785
I20260807 10:50:10.374377 9123 heartbeater.cc:464] Registering TS with master...
I20260807 10:50:10.375437 9123 heartbeater.cc:511] Master 127.3.73.254:32785 requested a full tablet report, sending...
I20260807 10:50:10.378038 8844 ts_manager.cc:194] Registered new tserver with Master: ae218d25dacb45dca53d947e3ca3339c (127.3.73.193:42059)
I20260807 10:50:10.379828 8844 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.193@KRBTEST.COM'} at 127.3.73.193:45589
WARNING: no policy specified for kudu/127.3.73.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab.
Entry for principal kudu/127.3.73.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab.
I20260807 10:50:10.419946 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.194:0
--local_ip_for_outbound_sockets=127.3.73.194
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:32785
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:44743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:50:10.778573 9133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:10.778875 9133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:10.779026 9133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:10.786492 9133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:10.786617 9133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:10.786839 9133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.194
I20260807 10:50:10.797080 9133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:44743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:32785
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.194
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:10.798491 9133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:10.799921 9133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:10.814574 9140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:10.814597 9141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:10.814554 9144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:10.815559 9133 server_base.cc:1034] running on GCE node
I20260807 10:50:10.816311 9133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:10.817503 9133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:10.818791 9133 hybrid_clock.cc:648] HybridClock initialized: now 1786099810818709 us; error 87 us; skew 500 ppm
I20260807 10:50:10.819182 9133 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:10 dist-test-slave-x5fp krb5kdc[8777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099810, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:10.822831 9133 init.cc:377] Logged in from keytab as kudu/127.3.73.194@KRBTEST.COM (short username kudu)
I20260807 10:50:10.824514 9133 webserver.cc:460] Webserver started at http://127.3.73.194:42439/ using document root <none> and password file <none>
I20260807 10:50:10.825322 9133 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:10.825479 9133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:10.825827 9133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:10.829171 9133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-1/data/instance:
uuid: "cb99a433194e466fb7a0a79c185483b4"
format_stamp: "Formatted at 2026-08-07 10:50:10 on dist-test-slave-x5fp"
server_key: "aa56a780ad2f5b55afcaf4456bc03d90"
server_key_iv: "abeac439917f3e55aa4a466a0a75296a"
server_key_version: "encryptionkey@0"
I20260807 10:50:10.830045 9133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance:
uuid: "cb99a433194e466fb7a0a79c185483b4"
format_stamp: "Formatted at 2026-08-07 10:50:10 on dist-test-slave-x5fp"
server_key: "aa56a780ad2f5b55afcaf4456bc03d90"
server_key_iv: "abeac439917f3e55aa4a466a0a75296a"
server_key_version: "encryptionkey@0"
I20260807 10:50:10.835796 9133 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 10:50:10.839574 9156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:10.841120 9133 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 10:50:10.841310 9133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-1/wal
uuid: "cb99a433194e466fb7a0a79c185483b4"
format_stamp: "Formatted at 2026-08-07 10:50:10 on dist-test-slave-x5fp"
server_key: "aa56a780ad2f5b55afcaf4456bc03d90"
server_key_iv: "abeac439917f3e55aa4a466a0a75296a"
server_key_version: "encryptionkey@0"
I20260807 10:50:10.841538 9133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:10.890995 9133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:10.896272 9133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:10.896631 9133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:10.897819 9133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:50:10.899320 9133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:50:10.899425 9133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:10.899582 9133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:50:10.899655 9133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:10.936013 9133 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.194:45973
I20260807 10:50:10.936056 9328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.194:45973 every 8 connection(s)
I20260807 10:50:10.937445 9133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
I20260807 10:50:10.938638 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 9133
I20260807 10:50:10.938908 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance
I20260807 10:50:10.939483 3367 external_mini_cluster.cc:1428] Setting key 807c8daa8705717f85e0de6f41ea17ba
Aug 07 10:50:10 dist-test-slave-x5fp krb5kdc[8777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099810, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:50:10.962594 9331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:10.940658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:36305 (local address 127.3.73.254:32785)
0807 10:50:10.941189 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:10.941196 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:10.942286 (+ 1090us) server_negotiation.cc:407] Connection header received
0807 10:50:10.944231 (+ 1945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:10.944239 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:10.944338 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:10.944464 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:10.946264 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:10.947514 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:10.949279 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:10.950058 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:10.950215 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:10.954434 (+ 4219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:10.954443 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:10.954451 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:10.954505 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:10.958600 (+ 4095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:10.959573 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:10.959582 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:10.959588 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:10.959772 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:10.960284 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:10.960293 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:10.960299 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:10.960709 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:10.960902 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:10.961721 (+ 819us) server_negotiation.cc:299] Negotiation successful
0807 10:50:10.962067 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":248,"threads_started":1}
I20260807 10:50:10.963323 9332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:10.941244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:32785 (local address 127.3.73.194:36305)
0807 10:50:10.942106 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:10.942161 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:10.943943 (+ 1782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:10.944754 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:10.944768 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:10.945366 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:10.946055 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:10.946073 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:10.947699 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:10.947703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:10.949126 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:10.949140 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:10.950333 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:10.950339 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:10.950568 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:10.951398 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:10.951423 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:10.954127 (+ 2704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:10.958840 (+ 4713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:10.958850 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:10.958865 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:10.959410 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:10.959897 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:10.959900 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:10.959903 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:10.960123 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:10.960889 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:10.960896 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:10.961275 (+ 379us) client_negotiation.cc:769] Sending connection context
0807 10:50:10.961542 (+ 267us) client_negotiation.cc:240] Negotiation successful
0807 10:50:10.962026 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":353,"threads_started":1}
I20260807 10:50:10.965145 9329 heartbeater.cc:347] Connected to a master server at 127.3.73.254:32785
I20260807 10:50:10.965518 9329 heartbeater.cc:464] Registering TS with master...
I20260807 10:50:10.966373 9329 heartbeater.cc:511] Master 127.3.73.254:32785 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.3.73.195@KRBTEST.COM; defaulting to no policy
I20260807 10:50:10.968173 8844 ts_manager.cc:194] Registered new tserver with Master: cb99a433194e466fb7a0a79c185483b4 (127.3.73.194:45973)
I20260807 10:50:10.969139 8844 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.194@KRBTEST.COM'} at 127.3.73.194:36305
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab.
Entry for principal kudu/127.3.73.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab.
I20260807 10:50:11.004668 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.195:0
--local_ip_for_outbound_sockets=127.3.73.195
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:32785
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:44743
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:50:11.367028 9337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:11.367447 9337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:11.367635 9337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:11.375141 9337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:11.375322 9337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:11.375517 9337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.195
I20260807 10:50:11.384325 9123 heartbeater.cc:503] Master 127.3.73.254:32785 was elected leader, sending a full tablet report...
I20260807 10:50:11.386649 9337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:44743
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:32785
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.195
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:11.388208 9337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:11.389809 9337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:11.404650 9346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:11.405120 9349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:11.405682 9347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:11.406241 9337 server_base.cc:1034] running on GCE node
I20260807 10:50:11.407042 9337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:11.408249 9337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:11.409524 9337 hybrid_clock.cc:648] HybridClock initialized: now 1786099811409472 us; error 43 us; skew 500 ppm
I20260807 10:50:11.410000 9337 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:11 dist-test-slave-x5fp krb5kdc[8777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099811, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:11.414080 9337 init.cc:377] Logged in from keytab as kudu/127.3.73.195@KRBTEST.COM (short username kudu)
I20260807 10:50:11.415990 9337 webserver.cc:460] Webserver started at http://127.3.73.195:46459/ using document root <none> and password file <none>
I20260807 10:50:11.416894 9337 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:11.417076 9337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:11.417461 9337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:11.420848 9337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-2/data/instance:
uuid: "b79e957d069b45aa8d353084d3a2995c"
format_stamp: "Formatted at 2026-08-07 10:50:11 on dist-test-slave-x5fp"
server_key: "1433ff59ce8e96f1ff25a5cf1e3d5394"
server_key_iv: "5421800fd7a9f64568b96c2ef1133871"
server_key_version: "encryptionkey@0"
I20260807 10:50:11.421759 9337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance:
uuid: "b79e957d069b45aa8d353084d3a2995c"
format_stamp: "Formatted at 2026-08-07 10:50:11 on dist-test-slave-x5fp"
server_key: "1433ff59ce8e96f1ff25a5cf1e3d5394"
server_key_iv: "5421800fd7a9f64568b96c2ef1133871"
server_key_version: "encryptionkey@0"
I20260807 10:50:11.427832 9337 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 10:50:11.431769 9357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:11.433259 9337 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 10:50:11.433463 9337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-2/wal
uuid: "b79e957d069b45aa8d353084d3a2995c"
format_stamp: "Formatted at 2026-08-07 10:50:11 on dist-test-slave-x5fp"
server_key: "1433ff59ce8e96f1ff25a5cf1e3d5394"
server_key_iv: "5421800fd7a9f64568b96c2ef1133871"
server_key_version: "encryptionkey@0"
I20260807 10:50:11.433705 9337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:11.448411 9337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:11.453171 9337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:11.453521 9337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:11.454612 9337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:50:11.456138 9337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:50:11.456248 9337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:11.456396 9337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:50:11.456472 9337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:11.494863 9337 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.195:42629
I20260807 10:50:11.494910 9527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.195:42629 every 8 connection(s)
I20260807 10:50:11.496269 9337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
I20260807 10:50:11.503041 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 9337
I20260807 10:50:11.503317 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance
I20260807 10:50:11.503849 3367 external_mini_cluster.cc:1428] Setting key 3e19d573e4a4bcdbd50f8fe5341779be
Aug 07 10:50:11 dist-test-slave-x5fp krb5kdc[8777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099811, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:50:11.518162 9532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:11.499092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:40455 (local address 127.3.73.254:32785)
0807 10:50:11.499577 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:11.499582 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:11.500591 (+ 1009us) server_negotiation.cc:407] Connection header received
0807 10:50:11.502618 (+ 2027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:11.502625 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:11.502715 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:11.502860 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:11.504782 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:11.505990 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:11.507753 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:11.508160 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:11.508269 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:11.512069 (+ 3800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:11.512075 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:11.512078 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:11.512116 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:11.514930 (+ 2814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:11.515686 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:11.515690 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:11.515693 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:11.515849 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:11.516294 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:11.516301 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:11.516304 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:11.516637 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:11.516842 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:11.517549 (+ 707us) server_negotiation.cc:299] Negotiation successful
0807 10:50:11.517770 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":243,"threads_started":1}
I20260807 10:50:11.519245 9533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:11.499524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:32785 (local address 127.3.73.195:40455)
0807 10:50:11.500308 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:11.500353 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:11.502304 (+ 1951us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:11.503141 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:11.503156 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:11.503806 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:11.504484 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:11.504511 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:11.506149 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:11.506153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:11.507578 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:11.507592 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:11.508363 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:11.508368 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:11.508570 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:11.509332 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:11.509356 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:11.511819 (+ 2463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:11.515086 (+ 3267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:11.515094 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:11.515108 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:11.515560 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:11.515951 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:11.515954 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:11.515957 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:11.516147 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:11.516838 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:11.516844 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:11.517189 (+ 345us) client_negotiation.cc:769] Sending connection context
0807 10:50:11.517426 (+ 237us) client_negotiation.cc:240] Negotiation successful
0807 10:50:11.517868 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":330,"threads_started":1}
I20260807 10:50:11.520570 9530 heartbeater.cc:347] Connected to a master server at 127.3.73.254:32785
I20260807 10:50:11.520954 9530 heartbeater.cc:464] Registering TS with master...
I20260807 10:50:11.521680 9530 heartbeater.cc:511] Master 127.3.73.254:32785 requested a full tablet report, sending...
I20260807 10:50:11.523203 8844 ts_manager.cc:194] Registered new tserver with Master: b79e957d069b45aa8d353084d3a2995c (127.3.73.195:42629)
I20260807 10:50:11.524027 8844 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.195@KRBTEST.COM'} at 127.3.73.195:40455
I20260807 10:50:11.532226 3367 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 10:50:11 dist-test-slave-x5fp krb5kdc[8777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 10:50:11 dist-test-slave-x5fp krb5kdc[8777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.193@KRBTEST.COM
I20260807 10:50:11.598593 9541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:11.582608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55572 (local address 127.3.73.193:42059)
0807 10:50:11.583208 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:11.583215 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:11.583243 (+ 28us) server_negotiation.cc:407] Connection header received
0807 10:50:11.583323 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:11.583330 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:11.583552 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:11.583692 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:11.585213 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:11.586246 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:11.587568 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:11.588158 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:11.588250 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:11.592433 (+ 4183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:11.592452 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:11.592468 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:11.592506 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:11.595656 (+ 3150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:11.596329 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:11.596336 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:11.596338 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:11.596466 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:11.596992 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:11.596996 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:11.596998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:11.597343 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:11.597581 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:11.598050 (+ 469us) server_negotiation.cc:299] Negotiation successful
0807 10:50:11.598255 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":295,"threads_started":1}
Aug 07 10:50:11 dist-test-slave-x5fp krb5kdc[8777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:50:11.618216 9532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:11.603583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33898 (local address 127.3.73.254:32785)
0807 10:50:11.603824 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:11.603831 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:11.604038 (+ 207us) server_negotiation.cc:407] Connection header received
0807 10:50:11.604246 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:11.604257 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:11.604336 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:11.604451 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:11.605937 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:11.606777 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:11.608130 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:11.608587 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:11.608756 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:11.613071 (+ 4315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:11.613076 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:11.613079 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:11.613114 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:11.615617 (+ 2503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:11.616291 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:11.616295 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:11.616305 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:11.616438 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:11.616954 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:11.616958 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:11.616961 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:11.617346 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:11.617533 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:11.617806 (+ 273us) server_negotiation.cc:299] Negotiation successful
0807 10:50:11.617976 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260807 10:50:11.618825 8844 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33898
I20260807 10:50:11.619809 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 8928
I20260807 10:50:11.632565 9541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:11.620695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55576 (local address 127.3.73.193:42059)
0807 10:50:11.621010 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:11.621016 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:11.621182 (+ 166us) server_negotiation.cc:407] Connection header received
0807 10:50:11.621344 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:11.621348 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:11.621438 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:11.621518 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:11.622693 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:11.623381 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:11.624620 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:11.625124 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:11.625233 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:11.627427 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:11.627431 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:11.627434 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:11.627465 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:11.630123 (+ 2658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:11.630744 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:11.630749 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:11.630753 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:11.630890 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:11.631290 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:11.631293 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:11.631295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:11.631545 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:11.631722 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:11.632006 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 10:50:11.632184 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165}
I20260807 10:50:11.633303 9117 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:11.718093 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 8928
I20260807 10:50:11.744972 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 9133
Aug 07 10:50:11 dist-test-slave-x5fp krb5kdc[8777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.194@KRBTEST.COM
I20260807 10:50:11.762481 9546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:11.746041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36536 (local address 127.3.73.194:45973)
0807 10:50:11.746635 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:11.746645 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:11.746679 (+ 34us) server_negotiation.cc:407] Connection header received
0807 10:50:11.746746 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:11.746752 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:11.746974 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:11.747131 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:11.748619 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:11.749843 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:11.751148 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:11.751710 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:11.751816 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:11.756210 (+ 4394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:11.756223 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:11.756237 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:11.756279 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:11.759429 (+ 3150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:11.760133 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:11.760141 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:11.760144 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:11.760289 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:11.760647 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:11.760651 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:11.760653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:11.761170 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:11.761440 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:11.761810 (+ 370us) server_negotiation.cc:299] Negotiation successful
0807 10:50:11.762095 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":316,"threads_started":1}
I20260807 10:50:11.763897 9323 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:11.849716 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 9133
I20260807 10:50:11.874939 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 9337
Aug 07 10:50:11 dist-test-slave-x5fp krb5kdc[8777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.195@KRBTEST.COM
I20260807 10:50:11.893116 9549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:11.876116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40116 (local address 127.3.73.195:42629)
0807 10:50:11.876818 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:11.876826 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:11.876857 (+ 31us) server_negotiation.cc:407] Connection header received
0807 10:50:11.876969 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:11.876975 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:11.877196 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:11.877326 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:11.878675 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:11.879744 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:11.881312 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:11.881980 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:11.882127 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:11.886740 (+ 4613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:11.886791 (+ 51us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:11.886809 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:11.886853 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:11.890099 (+ 3246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:11.890894 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:11.890903 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:11.890905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:11.891054 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:11.891477 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:11.891481 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:11.891483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:11.891878 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:11.892127 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:11.892395 (+ 268us) server_negotiation.cc:299] Negotiation successful
0807 10:50:11.892622 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":346,"threads_started":1}
I20260807 10:50:11.894572 9521 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:11.984016 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 9337
I20260807 10:50:12.010027 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 8804
I20260807 10:50:12.011346 8887 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:12.119969 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 8804
2026-08-07T10:50:12Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4585 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:50:12 dist-test-slave-x5fp krb5kdc[9555](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:50:12 dist-test-slave-x5fp krb5kdc[9555](info): set up 2 sockets
Aug 07 10:50:12 dist-test-slave-x5fp krb5kdc[9555](info): commencing operation
krb5kdc: starting...
W20260807 10:50:13.443374 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.000s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:50:13 dist-test-slave-x5fp krb5kdc[9555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:50:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:50:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.73.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
I20260807 10:50:13.721606 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:41915
--webserver_interface=127.3.73.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:43843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:41915
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:50:14.086253 9571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:14.086628 9571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:14.086870 9571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:14.095351 9571 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:50:14.095491 9571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:14.095582 9571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:14.095690 9571 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:50:14.095768 9571 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:50:14.107089 9571 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:43843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:41915
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:41915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:14.108568 9571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:14.110010 9571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:14.122709 9581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:14.122684 9584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:14.122620 9582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:14.126520 9571 server_base.cc:1034] running on GCE node
I20260807 10:50:14.127384 9571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:14.129043 9571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:14.130262 9571 hybrid_clock.cc:648] HybridClock initialized: now 1786099814130217 us; error 31 us; skew 500 ppm
I20260807 10:50:14.130645 9571 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:14 dist-test-slave-x5fp krb5kdc[9555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099814, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:14.134024 9571 init.cc:377] Logged in from keytab as kudu/127.3.73.254@KRBTEST.COM (short username kudu)
I20260807 10:50:14.135413 9571 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:14.135530 9571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:14.135887 9571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:14.139101 9571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "d7fef51c867947619557bc467c12ad35"
format_stamp: "Formatted at 2026-08-07 10:50:14 on dist-test-slave-x5fp"
server_key: "088cbaf19c3e5cb20e239758a8f4014f"
server_key_iv: "e280747d15559b245c07dae195ff2796"
server_key_version: "encryptionkey@0"
I20260807 10:50:14.139945 9571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "d7fef51c867947619557bc467c12ad35"
format_stamp: "Formatted at 2026-08-07 10:50:14 on dist-test-slave-x5fp"
server_key: "088cbaf19c3e5cb20e239758a8f4014f"
server_key_iv: "e280747d15559b245c07dae195ff2796"
server_key_version: "encryptionkey@0"
I20260807 10:50:14.145370 9571 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 10:50:14.149202 9593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:14.150568 9571 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 10:50:14.150749 9571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "d7fef51c867947619557bc467c12ad35"
format_stamp: "Formatted at 2026-08-07 10:50:14 on dist-test-slave-x5fp"
server_key: "088cbaf19c3e5cb20e239758a8f4014f"
server_key_iv: "e280747d15559b245c07dae195ff2796"
server_key_version: "encryptionkey@0"
I20260807 10:50:14.150988 9571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:14.180104 9571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:14.184557 9571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:14.184894 9571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:14.205514 9571 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:41915
I20260807 10:50:14.205494 9661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:41915 every 8 connection(s)
I20260807 10:50:14.206512 9571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:50:14.210616 9662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:14.210589 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 9571
I20260807 10:50:14.210853 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:50:14.211688 3367 external_mini_cluster.cc:1428] Setting key 22a690dbb61476982409bd7282de2b65
I20260807 10:50:14.223824 9662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7fef51c867947619557bc467c12ad35: Bootstrap starting.
Aug 07 10:50:14 dist-test-slave-x5fp krb5kdc[9555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:50:14.227434 9662 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7fef51c867947619557bc467c12ad35: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:14.228518 9662 log.cc:824] T 00000000000000000000000000000000 P d7fef51c867947619557bc467c12ad35: Log is configured to *not* fsync() on all Append() calls
I20260807 10:50:14.232012 9662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7fef51c867947619557bc467c12ad35: No bootstrap required, opened a new log
I20260807 10:50:14.236405 9665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:14.213925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44538 (local address 127.3.73.254:41915)
0807 10:50:14.214813 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:14.214825 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:14.214866 (+ 41us) server_negotiation.cc:407] Connection header received
0807 10:50:14.217560 (+ 2694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:14.217585 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:14.218014 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:14.218452 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:14.220059 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:14.221512 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:14.222824 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:14.223533 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:14.223721 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:14.228175 (+ 4454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:14.228184 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:14.228199 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:14.228241 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:14.231974 (+ 3733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:14.232925 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:14.232932 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:14.232938 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:14.233116 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:14.233547 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:14.233550 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:14.233552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:14.234136 (+ 584us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:14.234362 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:14.234927 (+ 565us) server_negotiation.cc:299] Negotiation successful
0807 10:50:14.235380 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":429,"threads_started":1}
I20260807 10:50:14.240671 9662 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7fef51c867947619557bc467c12ad35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7fef51c867947619557bc467c12ad35" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41915 } }
I20260807 10:50:14.241115 9662 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7fef51c867947619557bc467c12ad35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:14.241228 9662 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7fef51c867947619557bc467c12ad35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7fef51c867947619557bc467c12ad35, State: Initialized, Role: FOLLOWER
I20260807 10:50:14.241793 9662 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7fef51c867947619557bc467c12ad35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7fef51c867947619557bc467c12ad35" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41915 } }
I20260807 10:50:14.242043 9662 raft_consensus.cc:388] T 00000000000000000000000000000000 P d7fef51c867947619557bc467c12ad35 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:50:14.242200 9662 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7fef51c867947619557bc467c12ad35 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:50:14.242355 9662 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7fef51c867947619557bc467c12ad35 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:14.244593 9662 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7fef51c867947619557bc467c12ad35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7fef51c867947619557bc467c12ad35" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41915 } }
I20260807 10:50:14.245150 9662 leader_election.cc:302] T 00000000000000000000000000000000 P d7fef51c867947619557bc467c12ad35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d7fef51c867947619557bc467c12ad35; no voters:
I20260807 10:50:14.245822 9662 leader_election.cc:288] T 00000000000000000000000000000000 P d7fef51c867947619557bc467c12ad35 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:50:14.245992 9669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7fef51c867947619557bc467c12ad35 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:50:14.247457 9669 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7fef51c867947619557bc467c12ad35 [term 1 LEADER]: Becoming Leader. State: Replica: d7fef51c867947619557bc467c12ad35, State: Running, Role: LEADER
I20260807 10:50:14.248193 9669 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7fef51c867947619557bc467c12ad35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7fef51c867947619557bc467c12ad35" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41915 } }
I20260807 10:50:14.249312 9662 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7fef51c867947619557bc467c12ad35 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:50:14.251396 9675 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7fef51c867947619557bc467c12ad35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7fef51c867947619557bc467c12ad35. Latest consensus state: current_term: 1 leader_uuid: "d7fef51c867947619557bc467c12ad35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7fef51c867947619557bc467c12ad35" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41915 } } }
I20260807 10:50:14.251962 9675 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7fef51c867947619557bc467c12ad35 [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:14.254528 9679 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:50:14.256763 9673 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7fef51c867947619557bc467c12ad35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d7fef51c867947619557bc467c12ad35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7fef51c867947619557bc467c12ad35" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41915 } } }
I20260807 10:50:14.257347 9673 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7fef51c867947619557bc467c12ad35 [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:14.261406 9679 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:50:14.271199 9679 catalog_manager.cc:1347] Generated new cluster ID: d9838bcb9b144b61802e155e0c7b44aa
I20260807 10:50:14.271325 9679 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:50:14.281682 9679 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:50:14.283564 9679 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:50:14.302575 9679 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d7fef51c867947619557bc467c12ad35: Generated new TSK 0
I20260807 10:50:14.303519 9679 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.73.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab.
Entry for principal kudu/127.3.73.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab.
I20260807 10:50:14.381697 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.193:0
--local_ip_for_outbound_sockets=127.3.73.193
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:41915
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:43843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:50:14.751736 9703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:14.752087 9703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:14.752296 9703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:14.761178 9703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:14.761318 9703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:14.761536 9703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.193
I20260807 10:50:14.772575 9703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:43843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:41915
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.193
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:14.774114 9703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:14.775537 9703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:14.788059 9710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:14.788666 9711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:14.790086 9714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:14.790634 9703 server_base.cc:1034] running on GCE node
I20260807 10:50:14.791437 9703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:14.792511 9703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:14.793769 9703 hybrid_clock.cc:648] HybridClock initialized: now 1786099814793699 us; error 67 us; skew 500 ppm
I20260807 10:50:14.794144 9703 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:14 dist-test-slave-x5fp krb5kdc[9555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099814, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:14.797619 9703 init.cc:377] Logged in from keytab as kudu/127.3.73.193@KRBTEST.COM (short username kudu)
I20260807 10:50:14.798908 9703 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:14.799026 9703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:14.799376 9703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:14.802488 9703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-0/data/instance:
uuid: "1ae6373bba3643a783baf971c8c1af54"
format_stamp: "Formatted at 2026-08-07 10:50:14 on dist-test-slave-x5fp"
server_key: "fd1f7801868a2b7444cb83c3f5bf9f79"
server_key_iv: "1c4f425b45769eb81b300dfddb827a2b"
server_key_version: "encryptionkey@0"
I20260807 10:50:14.803364 9703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance:
uuid: "1ae6373bba3643a783baf971c8c1af54"
format_stamp: "Formatted at 2026-08-07 10:50:14 on dist-test-slave-x5fp"
server_key: "fd1f7801868a2b7444cb83c3f5bf9f79"
server_key_iv: "1c4f425b45769eb81b300dfddb827a2b"
server_key_version: "encryptionkey@0"
I20260807 10:50:14.808995 9703 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 10:50:14.813066 9731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:14.814541 9703 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 10:50:14.814734 9703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-0/wal
uuid: "1ae6373bba3643a783baf971c8c1af54"
format_stamp: "Formatted at 2026-08-07 10:50:14 on dist-test-slave-x5fp"
server_key: "fd1f7801868a2b7444cb83c3f5bf9f79"
server_key_iv: "1c4f425b45769eb81b300dfddb827a2b"
server_key_version: "encryptionkey@0"
I20260807 10:50:14.815004 9703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:14.840783 9703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:14.845599 9703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:14.845950 9703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:14.847110 9703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:50:14.848699 9703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:50:14.848868 9703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260807 10:50:14.849012 9703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:50:14.849109 9703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:14.889943 9703 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.193:36093
I20260807 10:50:14.890044 9896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.193:36093 every 8 connection(s)
I20260807 10:50:14.890986 9703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
I20260807 10:50:14.894897 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 9703
I20260807 10:50:14.895155 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance
I20260807 10:50:14.895955 3367 external_mini_cluster.cc:1428] Setting key d735522baca0015e6ee1a9e9df95b553
Aug 07 10:50:14 dist-test-slave-x5fp krb5kdc[9555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099814, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:50:14.912623 9901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:14.893812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:59581 (local address 127.3.73.254:41915)
0807 10:50:14.894416 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:14.894423 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:14.895519 (+ 1096us) server_negotiation.cc:407] Connection header received
0807 10:50:14.897589 (+ 2070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:14.897595 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:14.897689 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:14.897860 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:14.899439 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:14.900186 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:14.901966 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:14.902380 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:14.902530 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:14.906532 (+ 4002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:14.906538 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:14.906541 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:14.906579 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:14.909317 (+ 2738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:14.910149 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:14.910154 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:14.910156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:14.910285 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:14.910756 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:14.910759 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:14.910762 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:14.911035 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:14.911199 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:14.911974 (+ 775us) server_negotiation.cc:299] Negotiation successful
0807 10:50:14.912226 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":326,"threads_started":1}
I20260807 10:50:14.913831 9902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:14.894455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41915 (local address 127.3.73.193:59581)
0807 10:50:14.895308 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:14.895365 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:14.897313 (+ 1948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:14.898097 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:14.898106 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:14.898668 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:14.899251 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:14.899270 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:14.900359 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:14.900363 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:14.901760 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:14.901773 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:14.902629 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:14.902638 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:14.902884 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:14.903607 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:14.903633 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:14.906264 (+ 2631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:14.909493 (+ 3229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:14.909504 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:14.909523 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:14.910000 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:14.910403 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:14.910408 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:14.910411 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:14.910621 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:14.911171 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:14.911180 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:14.911567 (+ 387us) client_negotiation.cc:769] Sending connection context
0807 10:50:14.911850 (+ 283us) client_negotiation.cc:240] Negotiation successful
0807 10:50:14.912505 (+ 655us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":372,"threads_started":1}
I20260807 10:50:14.915763 9897 heartbeater.cc:347] Connected to a master server at 127.3.73.254:41915
I20260807 10:50:14.916152 9897 heartbeater.cc:464] Registering TS with master...
I20260807 10:50:14.917162 9897 heartbeater.cc:511] Master 127.3.73.254:41915 requested a full tablet report, sending...
I20260807 10:50:14.919957 9618 ts_manager.cc:194] Registered new tserver with Master: 1ae6373bba3643a783baf971c8c1af54 (127.3.73.193:36093)
I20260807 10:50:14.921530 9618 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.193@KRBTEST.COM'} at 127.3.73.193:59581
WARNING: no policy specified for kudu/127.3.73.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab.
Entry for principal kudu/127.3.73.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab.
I20260807 10:50:14.962304 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.194:0
--local_ip_for_outbound_sockets=127.3.73.194
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:41915
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:43843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:50:15.351951 9909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:15.352317 9909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:15.352509 9909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:15.360694 9909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:15.360922 9909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:15.361132 9909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.194
I20260807 10:50:15.371919 9909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:43843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:41915
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.194
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:15.373529 9909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:15.375043 9909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:15.389861 9918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:15.389894 9921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:15.390532 9917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:15.390921 9909 server_base.cc:1034] running on GCE node
I20260807 10:50:15.391535 9909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:15.392539 9909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:15.393831 9909 hybrid_clock.cc:648] HybridClock initialized: now 1786099815393801 us; error 89 us; skew 500 ppm
I20260807 10:50:15.394156 9909 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:15 dist-test-slave-x5fp krb5kdc[9555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099815, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:15.403317 9909 init.cc:377] Logged in from keytab as kudu/127.3.73.194@KRBTEST.COM (short username kudu)
I20260807 10:50:15.404910 9909 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:15.405071 9909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:15.405508 9909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:15.409613 9909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-1/data/instance:
uuid: "9640c7f8cdcd4990a999cecbe835e9a5"
format_stamp: "Formatted at 2026-08-07 10:50:15 on dist-test-slave-x5fp"
server_key: "fdd763da4e2d48247afe09bcfca59ea1"
server_key_iv: "229b5cbe3c8469230d5f70f73d89ee90"
server_key_version: "encryptionkey@0"
I20260807 10:50:15.410573 9909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance:
uuid: "9640c7f8cdcd4990a999cecbe835e9a5"
format_stamp: "Formatted at 2026-08-07 10:50:15 on dist-test-slave-x5fp"
server_key: "fdd763da4e2d48247afe09bcfca59ea1"
server_key_iv: "229b5cbe3c8469230d5f70f73d89ee90"
server_key_version: "encryptionkey@0"
I20260807 10:50:15.416601 9909 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 10:50:15.420816 9928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:15.422250 9909 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 10:50:15.422446 9909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-1/wal
uuid: "9640c7f8cdcd4990a999cecbe835e9a5"
format_stamp: "Formatted at 2026-08-07 10:50:15 on dist-test-slave-x5fp"
server_key: "fdd763da4e2d48247afe09bcfca59ea1"
server_key_iv: "229b5cbe3c8469230d5f70f73d89ee90"
server_key_version: "encryptionkey@0"
I20260807 10:50:15.422680 9909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:15.465867 9909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:15.470974 9909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:15.471305 9909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:15.472543 9909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:50:15.474139 9909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:50:15.474243 9909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:15.474396 9909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:50:15.474509 9909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:15.510712 9909 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.194:42957
I20260807 10:50:15.510764 10101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.194:42957 every 8 connection(s)
I20260807 10:50:15.511626 9909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
I20260807 10:50:15.517177 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 9909
I20260807 10:50:15.517398 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance
I20260807 10:50:15.517989 3367 external_mini_cluster.cc:1428] Setting key d7fd49f06407620e50d42396d68fb48b
Aug 07 10:50:15 dist-test-slave-x5fp krb5kdc[9555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099815, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:50:15.536132 10105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:15.514543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:53555 (local address 127.3.73.254:41915)
0807 10:50:15.515087 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:15.515094 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:15.516176 (+ 1082us) server_negotiation.cc:407] Connection header received
0807 10:50:15.518791 (+ 2615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:15.518799 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:15.518892 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:15.519021 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:15.521205 (+ 2184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:15.521933 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:15.524586 (+ 2653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:15.525153 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:15.525313 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:15.529556 (+ 4243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:15.529561 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:15.529564 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:15.529598 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:15.532900 (+ 3302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:15.533704 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:15.533708 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:15.533711 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:15.533838 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:15.534274 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:15.534280 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:15.534283 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:15.534573 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:15.534743 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:15.535484 (+ 741us) server_negotiation.cc:299] Negotiation successful
0807 10:50:15.535749 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":248,"threads_started":1}
I20260807 10:50:15.537307 10106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:15.515196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41915 (local address 127.3.73.194:53555)
0807 10:50:15.515997 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:15.516036 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:15.518429 (+ 2393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:15.519381 (+ 952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:15.519393 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:15.520149 (+ 756us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:15.521007 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:15.521038 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:15.522102 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:15.522110 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:15.524403 (+ 2293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:15.524429 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:15.525430 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:15.525440 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:15.525773 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:15.526784 (+ 1011us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:15.526810 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:15.529311 (+ 2501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:15.533049 (+ 3738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:15.533059 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:15.533077 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:15.533557 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:15.533952 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:15.533957 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:15.533961 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:15.534151 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:15.534706 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:15.534715 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:15.535083 (+ 368us) client_negotiation.cc:769] Sending connection context
0807 10:50:15.535370 (+ 287us) client_negotiation.cc:240] Negotiation successful
0807 10:50:15.535968 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":336,"threads_started":1}
I20260807 10:50:15.538894 10103 heartbeater.cc:347] Connected to a master server at 127.3.73.254:41915
I20260807 10:50:15.539253 10103 heartbeater.cc:464] Registering TS with master...
I20260807 10:50:15.540180 10103 heartbeater.cc:511] Master 127.3.73.254:41915 requested a full tablet report, sending...
I20260807 10:50:15.542023 9618 ts_manager.cc:194] Registered new tserver with Master: 9640c7f8cdcd4990a999cecbe835e9a5 (127.3.73.194:42957)
I20260807 10:50:15.543557 9618 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.194@KRBTEST.COM'} at 127.3.73.194:53555
WARNING: no policy specified for kudu/127.3.73.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab.
Entry for principal kudu/127.3.73.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab.
I20260807 10:50:15.584610 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.195:0
--local_ip_for_outbound_sockets=127.3.73.195
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:41915
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:43843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 10:50:15.925601 9897 heartbeater.cc:503] Master 127.3.73.254:41915 was elected leader, sending a full tablet report...
W20260807 10:50:15.931456 10109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:15.931788 10109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:15.931945 10109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:15.939452 10109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:15.939612 10109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:15.939826 10109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.195
I20260807 10:50:15.950474 10109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:43843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:41915
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.195
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:15.952137 10109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:15.953673 10109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:15.970249 10116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:15.970305 10117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:15.970980 10119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:15.972349 10109 server_base.cc:1034] running on GCE node
I20260807 10:50:15.973111 10109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:15.974097 10109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:15.975319 10109 hybrid_clock.cc:648] HybridClock initialized: now 1786099815975273 us; error 41 us; skew 500 ppm
I20260807 10:50:15.975649 10109 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:15 dist-test-slave-x5fp krb5kdc[9555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099815, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:15.979027 10109 init.cc:377] Logged in from keytab as kudu/127.3.73.195@KRBTEST.COM (short username kudu)
I20260807 10:50:15.980315 10109 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:15.980451 10109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:15.980841 10109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:15.983922 10109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-2/data/instance:
uuid: "b7186face0bc40c4a8c93f8b783cf276"
format_stamp: "Formatted at 2026-08-07 10:50:15 on dist-test-slave-x5fp"
server_key: "5faebc725de591f775eed10fd61f5730"
server_key_iv: "be4985a1967274c73e04175e202136a7"
server_key_version: "encryptionkey@0"
I20260807 10:50:15.984865 10109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance:
uuid: "b7186face0bc40c4a8c93f8b783cf276"
format_stamp: "Formatted at 2026-08-07 10:50:15 on dist-test-slave-x5fp"
server_key: "5faebc725de591f775eed10fd61f5730"
server_key_iv: "be4985a1967274c73e04175e202136a7"
server_key_version: "encryptionkey@0"
I20260807 10:50:15.990233 10109 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 10:50:15.994000 10129 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:15.995450 10109 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 10:50:15.995734 10109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-2/wal
uuid: "b7186face0bc40c4a8c93f8b783cf276"
format_stamp: "Formatted at 2026-08-07 10:50:15 on dist-test-slave-x5fp"
server_key: "5faebc725de591f775eed10fd61f5730"
server_key_iv: "be4985a1967274c73e04175e202136a7"
server_key_version: "encryptionkey@0"
I20260807 10:50:15.995998 10109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:16.014014 10109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:16.018805 10109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:16.019160 10109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:16.020241 10109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:50:16.021793 10109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:50:16.021920 10109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:16.022051 10109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:50:16.022140 10109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:16.058766 10109 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.195:38921
I20260807 10:50:16.058769 10290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.195:38921 every 8 connection(s)
I20260807 10:50:16.059698 10109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
I20260807 10:50:16.062207 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 10109
I20260807 10:50:16.062446 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance
I20260807 10:50:16.063099 3367 external_mini_cluster.cc:1428] Setting key 7584965877cfbbdd5fc4fb25fc357d1a
Aug 07 10:50:16 dist-test-slave-x5fp krb5kdc[9555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099815, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:50:16.082108 10295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:16.062660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:52507 (local address 127.3.73.254:41915)
0807 10:50:16.063224 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:16.063232 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:16.064798 (+ 1566us) server_negotiation.cc:407] Connection header received
0807 10:50:16.066392 (+ 1594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:16.066398 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:16.066487 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:16.066618 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:16.068211 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:16.069108 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:16.071323 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:16.071747 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:16.071857 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:16.075505 (+ 3648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:16.075529 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:16.075540 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:16.075588 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:16.078805 (+ 3217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:16.079577 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:16.079583 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:16.079587 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:16.079729 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:16.080166 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:16.080171 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:16.080175 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:16.080472 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:16.080648 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:16.081455 (+ 807us) server_negotiation.cc:299] Negotiation successful
0807 10:50:16.081695 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":251,"threads_started":1}
I20260807 10:50:16.083148 10296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:16.063264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41915 (local address 127.3.73.195:52507)
0807 10:50:16.064160 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:16.064217 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:16.066055 (+ 1838us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:16.066926 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:16.066934 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:16.067459 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:16.068022 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:16.068045 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:16.069351 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:16.069359 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:16.071152 (+ 1793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:16.071166 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:16.071970 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:16.071975 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:16.072164 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:16.072875 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:16.072898 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:16.075216 (+ 2318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:16.078964 (+ 3748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:16.078973 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:16.078991 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:16.079438 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:16.079842 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:16.079848 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:16.079851 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:16.080049 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:16.080622 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:16.080631 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:16.081073 (+ 442us) client_negotiation.cc:769] Sending connection context
0807 10:50:16.081360 (+ 287us) client_negotiation.cc:240] Negotiation successful
0807 10:50:16.081941 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":364,"threads_started":1}
I20260807 10:50:16.084955 10291 heartbeater.cc:347] Connected to a master server at 127.3.73.254:41915
I20260807 10:50:16.085292 10291 heartbeater.cc:464] Registering TS with master...
I20260807 10:50:16.086128 10291 heartbeater.cc:511] Master 127.3.73.254:41915 requested a full tablet report, sending...
I20260807 10:50:16.087797 9618 ts_manager.cc:194] Registered new tserver with Master: b7186face0bc40c4a8c93f8b783cf276 (127.3.73.195:38921)
I20260807 10:50:16.088621 9618 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.195@KRBTEST.COM'} at 127.3.73.195:52507
I20260807 10:50:16.091607 3367 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 10:50:16.111235 10295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:16.096416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44542 (local address 127.3.73.254:41915)
0807 10:50:16.096602 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:16.096606 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:16.097177 (+ 571us) server_negotiation.cc:407] Connection header received
0807 10:50:16.097298 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:16.097302 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:16.097370 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:16.097460 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:16.099500 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:16.100267 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:16.101716 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:16.102156 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:16.102300 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:16.104820 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:16.104826 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:16.104830 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:16.104866 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:16.107901 (+ 3035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:16.108755 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:16.108763 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:16.108768 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:16.108921 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:16.109433 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:16.109445 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:16.109449 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:16.109854 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:16.110160 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:16.110567 (+ 407us) server_negotiation.cc:299] Negotiation successful
0807 10:50:16.110851 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 10:50:16.115805 9618 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44542:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 10:50:16.123327 9618 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 10:50:16.167874 10311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:16.136888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58894 (local address 127.3.73.195:38921)
0807 10:50:16.149879 (+ 12991us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:16.149892 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:16.149932 (+ 40us) server_negotiation.cc:407] Connection header received
0807 10:50:16.150046 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:16.150059 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:16.151297 (+ 1238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:16.151537 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:50:16.153245 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:16.156363 (+ 3118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:16.162140 (+ 5777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:16.165819 (+ 3679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:16.165943 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:16.166311 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:16.167104 (+ 793us) server_negotiation.cc:299] Negotiation successful
0807 10:50:16.167300 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12739,"thread_start_us":14564,"threads_started":1}
I20260807 10:50:16.169272 10308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:16.136919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:42957 (local address 127.0.0.1:40268)
0807 10:50:16.152664 (+ 15745us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:16.152687 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:16.152965 (+ 278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:16.157271 (+ 4306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:16.157276 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:16.157307 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:50:16.159004 (+ 1697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:16.159024 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:16.162137 (+ 3113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:16.162142 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:16.165548 (+ 3406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:16.165565 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:16.168095 (+ 2530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:16.168105 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:16.168444 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:16.168520 (+ 76us) client_negotiation.cc:769] Sending connection context
0807 10:50:16.168663 (+ 143us) client_negotiation.cc:240] Negotiation successful
0807 10:50:16.168862 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15590,"mutex_wait_us":48,"thread_start_us":10995,"threads_started":1}
I20260807 10:50:16.169497 10312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:16.137568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40268 (local address 127.3.73.194:42957)
0807 10:50:16.156538 (+ 18970us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:16.156548 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:16.156579 (+ 31us) server_negotiation.cc:407] Connection header received
0807 10:50:16.156691 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:16.156700 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:16.157034 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:16.157292 (+ 258us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:50:16.159567 (+ 2275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:16.161970 (+ 2403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:16.165843 (+ 3873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:16.167668 (+ 1825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:16.168041 (+ 373us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:16.168272 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:16.168776 (+ 504us) server_negotiation.cc:299] Negotiation successful
0807 10:50:16.168986 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18746,"thread_start_us":19954,"threads_started":1}
I20260807 10:50:16.169917 10309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:16.136607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:38921 (local address 127.0.0.1:58894)
0807 10:50:16.148508 (+ 11901us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:16.148586 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:16.148866 (+ 280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:16.152359 (+ 3493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:16.152367 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:16.152428 (+ 61us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:50:16.152903 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:16.152918 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:16.156555 (+ 3637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:16.156563 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:16.161907 (+ 5344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:16.161924 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:16.166528 (+ 4604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:16.166534 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:16.166773 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:16.166830 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 10:50:16.168386 (+ 1556us) client_negotiation.cc:240] Negotiation successful
0807 10:50:16.168564 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11494,"thread_start_us":16434,"threads_started":1}
I20260807 10:50:16.173761 10001 tablet_service.cc:1467] Processing CreateTablet for tablet 66a0360af4b24a6c9f175a42e050f784 (DEFAULT_TABLE table=test-table [id=d045cf541a8e4ca099f6431b20f278be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:50:16.174492 10199 tablet_service.cc:1467] Processing CreateTablet for tablet 66a0360af4b24a6c9f175a42e050f784 (DEFAULT_TABLE table=test-table [id=d045cf541a8e4ca099f6431b20f278be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:50:16.175371 10001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66a0360af4b24a6c9f175a42e050f784. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:16.176097 10199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66a0360af4b24a6c9f175a42e050f784. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:16.179761 10317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:16.155803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:36093 (local address 127.0.0.1:41028)
0807 10:50:16.161629 (+ 5826us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:16.161651 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:16.161847 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:16.167763 (+ 5916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:16.167767 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:16.167796 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:50:16.168906 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:16.168921 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:16.172320 (+ 3399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:16.172325 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:16.175428 (+ 3103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:16.175446 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:16.178470 (+ 3024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:16.178478 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:16.178690 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:16.178728 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 10:50:16.178839 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 10:50:16.179016 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5645,"mutex_wait_us":40,"thread_start_us":5188,"threads_started":1}
I20260807 10:50:16.180960 10314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:16.149673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41028 (local address 127.3.73.193:36093)
0807 10:50:16.167039 (+ 17366us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:16.167050 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:16.167084 (+ 34us) server_negotiation.cc:407] Connection header received
0807 10:50:16.167190 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:16.167198 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:16.167516 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:16.168984 (+ 1468us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:50:16.169324 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:16.172110 (+ 2786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:16.175627 (+ 3517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:16.177804 (+ 2177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:16.177994 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:16.178257 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:16.179681 (+ 1424us) server_negotiation.cc:299] Negotiation successful
0807 10:50:16.180276 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17078,"thread_start_us":16766,"threads_started":1}
I20260807 10:50:16.186769 9801 tablet_service.cc:1467] Processing CreateTablet for tablet 66a0360af4b24a6c9f175a42e050f784 (DEFAULT_TABLE table=test-table [id=d045cf541a8e4ca099f6431b20f278be]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:50:16.188612 9801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 66a0360af4b24a6c9f175a42e050f784. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:16.196379 10318 tablet_bootstrap.cc:492] T 66a0360af4b24a6c9f175a42e050f784 P 9640c7f8cdcd4990a999cecbe835e9a5: Bootstrap starting.
I20260807 10:50:16.203982 10318 tablet_bootstrap.cc:654] T 66a0360af4b24a6c9f175a42e050f784 P 9640c7f8cdcd4990a999cecbe835e9a5: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:16.205241 10320 tablet_bootstrap.cc:492] T 66a0360af4b24a6c9f175a42e050f784 P b7186face0bc40c4a8c93f8b783cf276: Bootstrap starting.
I20260807 10:50:16.206017 10318 log.cc:824] T 66a0360af4b24a6c9f175a42e050f784 P 9640c7f8cdcd4990a999cecbe835e9a5: Log is configured to *not* fsync() on all Append() calls
I20260807 10:50:16.211016 10320 tablet_bootstrap.cc:654] T 66a0360af4b24a6c9f175a42e050f784 P b7186face0bc40c4a8c93f8b783cf276: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:16.213020 10320 log.cc:824] T 66a0360af4b24a6c9f175a42e050f784 P b7186face0bc40c4a8c93f8b783cf276: Log is configured to *not* fsync() on all Append() calls
I20260807 10:50:16.215051 10321 tablet_bootstrap.cc:492] T 66a0360af4b24a6c9f175a42e050f784 P 1ae6373bba3643a783baf971c8c1af54: Bootstrap starting.
I20260807 10:50:16.218711 10320 tablet_bootstrap.cc:492] T 66a0360af4b24a6c9f175a42e050f784 P b7186face0bc40c4a8c93f8b783cf276: No bootstrap required, opened a new log
I20260807 10:50:16.219043 10318 tablet_bootstrap.cc:492] T 66a0360af4b24a6c9f175a42e050f784 P 9640c7f8cdcd4990a999cecbe835e9a5: No bootstrap required, opened a new log
I20260807 10:50:16.219074 10320 ts_tablet_manager.cc:1397] T 66a0360af4b24a6c9f175a42e050f784 P b7186face0bc40c4a8c93f8b783cf276: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.008s
I20260807 10:50:16.219388 10318 ts_tablet_manager.cc:1397] T 66a0360af4b24a6c9f175a42e050f784 P 9640c7f8cdcd4990a999cecbe835e9a5: Time spent bootstrapping tablet: real 0.023s user 0.021s sys 0.000s
I20260807 10:50:16.219805 10321 tablet_bootstrap.cc:654] T 66a0360af4b24a6c9f175a42e050f784 P 1ae6373bba3643a783baf971c8c1af54: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:16.221446 10321 log.cc:824] T 66a0360af4b24a6c9f175a42e050f784 P 1ae6373bba3643a783baf971c8c1af54: Log is configured to *not* fsync() on all Append() calls
I20260807 10:50:16.231252 10320 raft_consensus.cc:348] T 66a0360af4b24a6c9f175a42e050f784 P b7186face0bc40c4a8c93f8b783cf276 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7186face0bc40c4a8c93f8b783cf276" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 38921 } } peers { permanent_uuid: "9640c7f8cdcd4990a999cecbe835e9a5" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 42957 } } peers { permanent_uuid: "1ae6373bba3643a783baf971c8c1af54" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 36093 } }
I20260807 10:50:16.231606 10321 tablet_bootstrap.cc:492] T 66a0360af4b24a6c9f175a42e050f784 P 1ae6373bba3643a783baf971c8c1af54: No bootstrap required, opened a new log
I20260807 10:50:16.231685 10320 raft_consensus.cc:374] T 66a0360af4b24a6c9f175a42e050f784 P b7186face0bc40c4a8c93f8b783cf276 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:16.231784 10320 raft_consensus.cc:729] T 66a0360af4b24a6c9f175a42e050f784 P b7186face0bc40c4a8c93f8b783cf276 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7186face0bc40c4a8c93f8b783cf276, State: Initialized, Role: FOLLOWER
I20260807 10:50:16.231931 10321 ts_tablet_manager.cc:1397] T 66a0360af4b24a6c9f175a42e050f784 P 1ae6373bba3643a783baf971c8c1af54: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.006s
I20260807 10:50:16.232453 10320 consensus_queue.cc:260] T 66a0360af4b24a6c9f175a42e050f784 P b7186face0bc40c4a8c93f8b783cf276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7186face0bc40c4a8c93f8b783cf276" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 38921 } } peers { permanent_uuid: "9640c7f8cdcd4990a999cecbe835e9a5" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 42957 } } peers { permanent_uuid: "1ae6373bba3643a783baf971c8c1af54" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 36093 } }
I20260807 10:50:16.233806 10291 heartbeater.cc:503] Master 127.3.73.254:41915 was elected leader, sending a full tablet report...
I20260807 10:50:16.234269 10320 ts_tablet_manager.cc:1428] T 66a0360af4b24a6c9f175a42e050f784 P b7186face0bc40c4a8c93f8b783cf276: Time spent starting tablet: real 0.015s user 0.010s sys 0.002s
I20260807 10:50:16.233347 10318 raft_consensus.cc:348] T 66a0360af4b24a6c9f175a42e050f784 P 9640c7f8cdcd4990a999cecbe835e9a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7186face0bc40c4a8c93f8b783cf276" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 38921 } } peers { permanent_uuid: "9640c7f8cdcd4990a999cecbe835e9a5" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 42957 } } peers { permanent_uuid: "1ae6373bba3643a783baf971c8c1af54" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 36093 } }
I20260807 10:50:16.234748 10318 raft_consensus.cc:374] T 66a0360af4b24a6c9f175a42e050f784 P 9640c7f8cdcd4990a999cecbe835e9a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:16.234869 10318 raft_consensus.cc:729] T 66a0360af4b24a6c9f175a42e050f784 P 9640c7f8cdcd4990a999cecbe835e9a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9640c7f8cdcd4990a999cecbe835e9a5, State: Initialized, Role: FOLLOWER
I20260807 10:50:16.235489 10318 consensus_queue.cc:260] T 66a0360af4b24a6c9f175a42e050f784 P 9640c7f8cdcd4990a999cecbe835e9a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7186face0bc40c4a8c93f8b783cf276" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 38921 } } peers { permanent_uuid: "9640c7f8cdcd4990a999cecbe835e9a5" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 42957 } } peers { permanent_uuid: "1ae6373bba3643a783baf971c8c1af54" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 36093 } }
I20260807 10:50:16.237531 10103 heartbeater.cc:503] Master 127.3.73.254:41915 was elected leader, sending a full tablet report...
I20260807 10:50:16.239094 10318 ts_tablet_manager.cc:1428] T 66a0360af4b24a6c9f175a42e050f784 P 9640c7f8cdcd4990a999cecbe835e9a5: Time spent starting tablet: real 0.020s user 0.011s sys 0.007s
I20260807 10:50:16.242600 10321 raft_consensus.cc:348] T 66a0360af4b24a6c9f175a42e050f784 P 1ae6373bba3643a783baf971c8c1af54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7186face0bc40c4a8c93f8b783cf276" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 38921 } } peers { permanent_uuid: "9640c7f8cdcd4990a999cecbe835e9a5" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 42957 } } peers { permanent_uuid: "1ae6373bba3643a783baf971c8c1af54" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 36093 } }
I20260807 10:50:16.243110 10321 raft_consensus.cc:374] T 66a0360af4b24a6c9f175a42e050f784 P 1ae6373bba3643a783baf971c8c1af54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:16.243243 10321 raft_consensus.cc:729] T 66a0360af4b24a6c9f175a42e050f784 P 1ae6373bba3643a783baf971c8c1af54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ae6373bba3643a783baf971c8c1af54, State: Initialized, Role: FOLLOWER
I20260807 10:50:16.243870 10321 consensus_queue.cc:260] T 66a0360af4b24a6c9f175a42e050f784 P 1ae6373bba3643a783baf971c8c1af54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7186face0bc40c4a8c93f8b783cf276" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 38921 } } peers { permanent_uuid: "9640c7f8cdcd4990a999cecbe835e9a5" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 42957 } } peers { permanent_uuid: "1ae6373bba3643a783baf971c8c1af54" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 36093 } }
I20260807 10:50:16.245669 10321 ts_tablet_manager.cc:1428] T 66a0360af4b24a6c9f175a42e050f784 P 1ae6373bba3643a783baf971c8c1af54: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260807 10:50:16.256950 10326 raft_consensus.cc:482] T 66a0360af4b24a6c9f175a42e050f784 P 9640c7f8cdcd4990a999cecbe835e9a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 10:50:16.257334 10326 raft_consensus.cc:504] T 66a0360af4b24a6c9f175a42e050f784 P 9640c7f8cdcd4990a999cecbe835e9a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7186face0bc40c4a8c93f8b783cf276" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 38921 } } peers { permanent_uuid: "9640c7f8cdcd4990a999cecbe835e9a5" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 42957 } } peers { permanent_uuid: "1ae6373bba3643a783baf971c8c1af54" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 36093 } }
I20260807 10:50:16.260970 10326 leader_election.cc:288] T 66a0360af4b24a6c9f175a42e050f784 P 9640c7f8cdcd4990a999cecbe835e9a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7186face0bc40c4a8c93f8b783cf276 (127.3.73.195:38921), 1ae6373bba3643a783baf971c8c1af54 (127.3.73.193:36093)
W20260807 10:50:16.263622 10104 tablet.cc:2369] T 66a0360af4b24a6c9f175a42e050f784 P 9640c7f8cdcd4990a999cecbe835e9a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:50:16.270572 10314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:16.260297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:50791 (local address 127.3.73.193:36093)
0807 10:50:16.260644 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:16.260654 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:16.262152 (+ 1498us) server_negotiation.cc:407] Connection header received
0807 10:50:16.262392 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:16.262404 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:16.262476 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:16.262594 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:50:16.263415 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:16.264251 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:16.267928 (+ 3677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:16.269268 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:16.269447 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:16.269527 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:16.270088 (+ 561us) server_negotiation.cc:299] Negotiation successful
0807 10:50:16.270250 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151}
I20260807 10:50:16.270571 10311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:16.260249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:51851 (local address 127.3.73.195:38921)
0807 10:50:16.260535 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:16.260548 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:16.262093 (+ 1545us) server_negotiation.cc:407] Connection header received
0807 10:50:16.262320 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:16.262326 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:16.262395 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:16.262491 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:50:16.263148 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:16.264249 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:16.267931 (+ 3682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:16.269272 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:16.269447 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:16.269528 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:16.270088 (+ 560us) server_negotiation.cc:299] Negotiation successful
0807 10:50:16.270250 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260807 10:50:16.270753 10330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:16.260226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:36093 (local address 127.3.73.194:50791)
0807 10:50:16.262021 (+ 1795us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:16.262039 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:16.262230 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:16.262616 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:16.262621 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:16.262671 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:50:16.263215 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:16.263226 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:16.264492 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:16.264498 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:16.267708 (+ 3210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:16.267729 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:16.269598 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:16.269603 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:16.269865 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:16.269915 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 10:50:16.270011 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 10:50:16.270239 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1675,"mutex_wait_us":316,"thread_start_us":1046,"threads_started":1}
I20260807 10:50:16.270807 10331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:16.260053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:38921 (local address 127.3.73.194:51851)
0807 10:50:16.261946 (+ 1893us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:16.261984 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:16.262161 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:16.262510 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:16.262514 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:16.262538 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:50:16.262982 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:16.262993 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:16.264414 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:16.264425 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:16.267708 (+ 3283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:16.267729 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:16.269573 (+ 1844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:16.269585 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:16.269791 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:16.269902 (+ 111us) client_negotiation.cc:769] Sending connection context
0807 10:50:16.270011 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 10:50:16.270247 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1730,"mutex_wait_us":266,"thread_start_us":1613,"threads_started":1}
I20260807 10:50:16.272202 10228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66a0360af4b24a6c9f175a42e050f784" candidate_uuid: "9640c7f8cdcd4990a999cecbe835e9a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7186face0bc40c4a8c93f8b783cf276" is_pre_election: true
I20260807 10:50:16.272202 9828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66a0360af4b24a6c9f175a42e050f784" candidate_uuid: "9640c7f8cdcd4990a999cecbe835e9a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ae6373bba3643a783baf971c8c1af54" is_pre_election: true
I20260807 10:50:16.272938 10228 raft_consensus.cc:2454] T 66a0360af4b24a6c9f175a42e050f784 P b7186face0bc40c4a8c93f8b783cf276 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9640c7f8cdcd4990a999cecbe835e9a5 in term 0.
I20260807 10:50:16.272938 9828 raft_consensus.cc:2454] T 66a0360af4b24a6c9f175a42e050f784 P 1ae6373bba3643a783baf971c8c1af54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9640c7f8cdcd4990a999cecbe835e9a5 in term 0.
I20260807 10:50:16.273754 9929 leader_election.cc:302] T 66a0360af4b24a6c9f175a42e050f784 P 9640c7f8cdcd4990a999cecbe835e9a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ae6373bba3643a783baf971c8c1af54, 9640c7f8cdcd4990a999cecbe835e9a5; no voters:
I20260807 10:50:16.274183 10326 raft_consensus.cc:2781] T 66a0360af4b24a6c9f175a42e050f784 P 9640c7f8cdcd4990a999cecbe835e9a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 10:50:16.274395 10326 raft_consensus.cc:482] T 66a0360af4b24a6c9f175a42e050f784 P 9640c7f8cdcd4990a999cecbe835e9a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 10:50:16.274535 10326 raft_consensus.cc:3037] T 66a0360af4b24a6c9f175a42e050f784 P 9640c7f8cdcd4990a999cecbe835e9a5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:16.277377 10326 raft_consensus.cc:504] T 66a0360af4b24a6c9f175a42e050f784 P 9640c7f8cdcd4990a999cecbe835e9a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7186face0bc40c4a8c93f8b783cf276" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 38921 } } peers { permanent_uuid: "9640c7f8cdcd4990a999cecbe835e9a5" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 42957 } } peers { permanent_uuid: "1ae6373bba3643a783baf971c8c1af54" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 36093 } }
I20260807 10:50:16.278182 10326 leader_election.cc:288] T 66a0360af4b24a6c9f175a42e050f784 P 9640c7f8cdcd4990a999cecbe835e9a5 [CANDIDATE]: Term 1 election: Requested vote from peers b7186face0bc40c4a8c93f8b783cf276 (127.3.73.195:38921), 1ae6373bba3643a783baf971c8c1af54 (127.3.73.193:36093)
I20260807 10:50:16.278653 10228 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66a0360af4b24a6c9f175a42e050f784" candidate_uuid: "9640c7f8cdcd4990a999cecbe835e9a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7186face0bc40c4a8c93f8b783cf276"
I20260807 10:50:16.278769 9828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "66a0360af4b24a6c9f175a42e050f784" candidate_uuid: "9640c7f8cdcd4990a999cecbe835e9a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ae6373bba3643a783baf971c8c1af54"
I20260807 10:50:16.278967 9828 raft_consensus.cc:3037] T 66a0360af4b24a6c9f175a42e050f784 P 1ae6373bba3643a783baf971c8c1af54 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:16.278949 10228 raft_consensus.cc:3037] T 66a0360af4b24a6c9f175a42e050f784 P b7186face0bc40c4a8c93f8b783cf276 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:16.282511 10228 raft_consensus.cc:2454] T 66a0360af4b24a6c9f175a42e050f784 P b7186face0bc40c4a8c93f8b783cf276 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9640c7f8cdcd4990a999cecbe835e9a5 in term 1.
I20260807 10:50:16.282511 9828 raft_consensus.cc:2454] T 66a0360af4b24a6c9f175a42e050f784 P 1ae6373bba3643a783baf971c8c1af54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9640c7f8cdcd4990a999cecbe835e9a5 in term 1.
I20260807 10:50:16.283030 9932 leader_election.cc:302] T 66a0360af4b24a6c9f175a42e050f784 P 9640c7f8cdcd4990a999cecbe835e9a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9640c7f8cdcd4990a999cecbe835e9a5, b7186face0bc40c4a8c93f8b783cf276; no voters:
I20260807 10:50:16.283439 10326 raft_consensus.cc:2781] T 66a0360af4b24a6c9f175a42e050f784 P 9640c7f8cdcd4990a999cecbe835e9a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:50:16.284149 10326 raft_consensus.cc:686] T 66a0360af4b24a6c9f175a42e050f784 P 9640c7f8cdcd4990a999cecbe835e9a5 [term 1 LEADER]: Becoming Leader. State: Replica: 9640c7f8cdcd4990a999cecbe835e9a5, State: Running, Role: LEADER
I20260807 10:50:16.285060 10326 consensus_queue.cc:237] T 66a0360af4b24a6c9f175a42e050f784 P 9640c7f8cdcd4990a999cecbe835e9a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7186face0bc40c4a8c93f8b783cf276" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 38921 } } peers { permanent_uuid: "9640c7f8cdcd4990a999cecbe835e9a5" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 42957 } } peers { permanent_uuid: "1ae6373bba3643a783baf971c8c1af54" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 36093 } }
I20260807 10:50:16.290201 9617 catalog_manager.cc:5495] T 66a0360af4b24a6c9f175a42e050f784 P 9640c7f8cdcd4990a999cecbe835e9a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9640c7f8cdcd4990a999cecbe835e9a5 (127.3.73.194). New cstate: current_term: 1 leader_uuid: "9640c7f8cdcd4990a999cecbe835e9a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7186face0bc40c4a8c93f8b783cf276" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 38921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9640c7f8cdcd4990a999cecbe835e9a5" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 42957 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ae6373bba3643a783baf971c8c1af54" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 36093 } health_report { overall_health: UNKNOWN } } }
W20260807 10:50:16.311507 10294 tablet.cc:2369] T 66a0360af4b24a6c9f175a42e050f784 P b7186face0bc40c4a8c93f8b783cf276: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:50:16.329030 10312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:16.322365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40282 (local address 127.3.73.194:42957)
0807 10:50:16.322576 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:16.322581 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:16.322710 (+ 129us) server_negotiation.cc:407] Connection header received
0807 10:50:16.322981 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:16.322986 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:16.323093 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:16.323232 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:50:16.323651 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:16.324538 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:16.326207 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:16.326865 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:16.326982 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:16.327618 (+ 636us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:50:16.328372 (+ 754us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:50:16.328494 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:16.328647 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 10:50:16.328763 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 10:50:16.338060 10228 raft_consensus.cc:1261] T 66a0360af4b24a6c9f175a42e050f784 P b7186face0bc40c4a8c93f8b783cf276 [term 1 FOLLOWER]: Refusing update from remote peer 9640c7f8cdcd4990a999cecbe835e9a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:50:16.338060 9828 raft_consensus.cc:1261] T 66a0360af4b24a6c9f175a42e050f784 P 1ae6373bba3643a783baf971c8c1af54 [term 1 FOLLOWER]: Refusing update from remote peer 9640c7f8cdcd4990a999cecbe835e9a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:50:16.339080 10332 consensus_queue.cc:1035] T 66a0360af4b24a6c9f175a42e050f784 P 9640c7f8cdcd4990a999cecbe835e9a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7186face0bc40c4a8c93f8b783cf276" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 38921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:50:16.339530 10326 consensus_queue.cc:1035] T 66a0360af4b24a6c9f175a42e050f784 P 9640c7f8cdcd4990a999cecbe835e9a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ae6373bba3643a783baf971c8c1af54" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 36093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:50:16.371492 9615 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:50:16.371778 9615 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:50:16.376164 9615 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d7fef51c867947619557bc467c12ad35: Sending DeleteTablet for 3 replicas of tablet 66a0360af4b24a6c9f175a42e050f784
I20260807 10:50:16.377585 10199 tablet_service.cc:1514] Processing DeleteTablet for tablet 66a0360af4b24a6c9f175a42e050f784 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:50:16 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:58894
I20260807 10:50:16.377637 10001 tablet_service.cc:1514] Processing DeleteTablet for tablet 66a0360af4b24a6c9f175a42e050f784 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:50:16 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:40268
I20260807 10:50:16.378456 9801 tablet_service.cc:1514] Processing DeleteTablet for tablet 66a0360af4b24a6c9f175a42e050f784 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:50:16 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:41028
I20260807 10:50:16.379462 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 9703
I20260807 10:50:16.392872 10354 tablet_replica.cc:330] T 66a0360af4b24a6c9f175a42e050f784 P b7186face0bc40c4a8c93f8b783cf276: stopping tablet replica
I20260807 10:50:16.393479 10354 raft_consensus.cc:2229] T 66a0360af4b24a6c9f175a42e050f784 P b7186face0bc40c4a8c93f8b783cf276 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:50:16.393860 10356 tablet_replica.cc:330] T 66a0360af4b24a6c9f175a42e050f784 P 1ae6373bba3643a783baf971c8c1af54: stopping tablet replica
I20260807 10:50:16.393977 10353 tablet_replica.cc:330] T 66a0360af4b24a6c9f175a42e050f784 P 9640c7f8cdcd4990a999cecbe835e9a5: stopping tablet replica
I20260807 10:50:16.394269 10354 raft_consensus.cc:2258] T 66a0360af4b24a6c9f175a42e050f784 P b7186face0bc40c4a8c93f8b783cf276 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:50:16.394654 10353 raft_consensus.cc:2229] T 66a0360af4b24a6c9f175a42e050f784 P 9640c7f8cdcd4990a999cecbe835e9a5 [term 1 LEADER]: Raft consensus shutting down.
I20260807 10:50:16.395159 10356 raft_consensus.cc:2229] T 66a0360af4b24a6c9f175a42e050f784 P 1ae6373bba3643a783baf971c8c1af54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:50:16.395533 10353 raft_consensus.cc:2258] T 66a0360af4b24a6c9f175a42e050f784 P 9640c7f8cdcd4990a999cecbe835e9a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:50:16.396703 10356 raft_consensus.cc:2258] T 66a0360af4b24a6c9f175a42e050f784 P 1ae6373bba3643a783baf971c8c1af54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:50:16.399091 10354 ts_tablet_manager.cc:1905] T 66a0360af4b24a6c9f175a42e050f784 P b7186face0bc40c4a8c93f8b783cf276: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:50:16.401777 10353 ts_tablet_manager.cc:1905] T 66a0360af4b24a6c9f175a42e050f784 P 9640c7f8cdcd4990a999cecbe835e9a5: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 10:50:16 dist-test-slave-x5fp krb5kdc[9555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.193@KRBTEST.COM
I20260807 10:50:16.405607 10356 ts_tablet_manager.cc:1905] T 66a0360af4b24a6c9f175a42e050f784 P 1ae6373bba3643a783baf971c8c1af54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:50:16.407186 10354 ts_tablet_manager.cc:1918] T 66a0360af4b24a6c9f175a42e050f784 P b7186face0bc40c4a8c93f8b783cf276: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:50:16.407405 10354 log.cc:1196] T 66a0360af4b24a6c9f175a42e050f784 P b7186face0bc40c4a8c93f8b783cf276: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-2/wal/wals/66a0360af4b24a6c9f175a42e050f784
I20260807 10:50:16.408061 10354 ts_tablet_manager.cc:1939] T 66a0360af4b24a6c9f175a42e050f784 P b7186face0bc40c4a8c93f8b783cf276: Deleting consensus metadata
I20260807 10:50:16.410374 9598 catalog_manager.cc:4841] TS b7186face0bc40c4a8c93f8b783cf276 (127.3.73.195:38921): tablet 66a0360af4b24a6c9f175a42e050f784 (table test-table [id=d045cf541a8e4ca099f6431b20f278be]) successfully deleted
I20260807 10:50:16.413461 10356 ts_tablet_manager.cc:1918] T 66a0360af4b24a6c9f175a42e050f784 P 1ae6373bba3643a783baf971c8c1af54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:50:16.413654 10356 log.cc:1196] T 66a0360af4b24a6c9f175a42e050f784 P 1ae6373bba3643a783baf971c8c1af54: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-0/wal/wals/66a0360af4b24a6c9f175a42e050f784
I20260807 10:50:16.414242 10356 ts_tablet_manager.cc:1939] T 66a0360af4b24a6c9f175a42e050f784 P 1ae6373bba3643a783baf971c8c1af54: Deleting consensus metadata
I20260807 10:50:16.414898 10353 ts_tablet_manager.cc:1918] T 66a0360af4b24a6c9f175a42e050f784 P 9640c7f8cdcd4990a999cecbe835e9a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:50:16.415107 10353 log.cc:1196] T 66a0360af4b24a6c9f175a42e050f784 P 9640c7f8cdcd4990a999cecbe835e9a5: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786099767749326-3367-0/minicluster-data/ts-1/wal/wals/66a0360af4b24a6c9f175a42e050f784
I20260807 10:50:16.415428 10314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:16.381331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41042 (local address 127.3.73.193:36093)
0807 10:50:16.381504 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:16.381513 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:16.391743 (+ 10230us) server_negotiation.cc:407] Connection header received
0807 10:50:16.391998 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:16.392004 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:16.392113 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:16.392222 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:16.396031 (+ 3809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:16.397425 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:16.398893 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:16.399588 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:16.399726 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:16.406093 (+ 6367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:16.406108 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:16.406131 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:16.406210 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:16.411356 (+ 5146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:16.412430 (+ 1074us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:16.412440 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:16.412444 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:16.412623 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:16.413157 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:16.413165 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:16.413171 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:16.413714 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:16.414087 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:16.414691 (+ 604us) server_negotiation.cc:299] Negotiation successful
0807 10:50:16.414964 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 10:50:16.415732 10353 ts_tablet_manager.cc:1939] T 66a0360af4b24a6c9f175a42e050f784 P 9640c7f8cdcd4990a999cecbe835e9a5: Deleting consensus metadata
I20260807 10:50:16.416337 9594 catalog_manager.cc:4841] TS 1ae6373bba3643a783baf971c8c1af54 (127.3.73.193:36093): tablet 66a0360af4b24a6c9f175a42e050f784 (table test-table [id=d045cf541a8e4ca099f6431b20f278be]) successfully deleted
I20260807 10:50:16.416999 9891 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:16.417680 9594 catalog_manager.cc:4841] TS 9640c7f8cdcd4990a999cecbe835e9a5 (127.3.73.194:42957): tablet 66a0360af4b24a6c9f175a42e050f784 (table test-table [id=d045cf541a8e4ca099f6431b20f278be]) successfully deleted
I20260807 10:50:16.509522 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 9703
I20260807 10:50:16.545962 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 9909
Aug 07 10:50:16 dist-test-slave-x5fp krb5kdc[9555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.194@KRBTEST.COM
I20260807 10:50:16.564517 10312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:16.546834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40288 (local address 127.3.73.194:42957)
0807 10:50:16.547046 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:16.547053 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:16.547076 (+ 23us) server_negotiation.cc:407] Connection header received
0807 10:50:16.547218 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:16.547223 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:16.547309 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:16.547428 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:16.548694 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:16.550030 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:16.551438 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:16.552139 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:16.552298 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:16.556788 (+ 4490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:16.556807 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:16.556828 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:16.556885 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:16.561319 (+ 4434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:16.562051 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:16.562062 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:16.562066 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:16.562259 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:16.562714 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:16.562719 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:16.562723 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:16.563249 (+ 526us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:16.563615 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:16.563851 (+ 236us) server_negotiation.cc:299] Negotiation successful
0807 10:50:16.564098 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 10:50:16.565382 10089 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:16.679593 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 9909
I20260807 10:50:16.708455 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 10109
Aug 07 10:50:16 dist-test-slave-x5fp krb5kdc[9555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099813, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.195@KRBTEST.COM
I20260807 10:50:16.724246 10311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:16.709288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58910 (local address 127.3.73.195:38921)
0807 10:50:16.709460 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:16.709465 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:16.709489 (+ 24us) server_negotiation.cc:407] Connection header received
0807 10:50:16.709675 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:16.709679 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:16.709750 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:16.709840 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:16.711159 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:16.711948 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:16.713434 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:16.713859 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:16.713964 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:16.718539 (+ 4575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:16.718558 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:16.718575 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:16.718617 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:16.721706 (+ 3089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:16.722361 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:16.722371 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:16.722374 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:16.722518 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:16.722953 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:16.722957 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:16.722959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:16.723315 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:16.723539 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:16.723810 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 10:50:16.723992 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 10:50:16.724977 10284 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:16.816246 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 10109
I20260807 10:50:16.841917 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 9571
I20260807 10:50:16.843108 9657 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:16.958184 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 9571
2026-08-07T10:50:16Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4833 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T10:50:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:50:17Z Disabled control of system clock
I20260807 10:50:17.045130 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:34993
--webserver_interface=127.3.73.254
--webserver_port=0
--builtin_ntp_servers=127.3.73.212:40099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:34993
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 10:50:17.401484 10367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:17.401816 10367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:17.401968 10367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:17.409458 10367 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:50:17.409577 10367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:17.409655 10367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:17.409749 10367 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:50:17.409850 10367 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:50:17.419436 10367 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:40099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:34993
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:34993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:17.420886 10367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:17.422770 10367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:17.435283 10374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:17.435216 10375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:17.435252 10378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:17.436069 10367 server_base.cc:1034] running on GCE node
I20260807 10:50:17.436919 10367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:17.438319 10367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:17.439527 10367 hybrid_clock.cc:648] HybridClock initialized: now 1786099817439483 us; error 31 us; skew 500 ppm
I20260807 10:50:17.439859 10367 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 10:50:17.442224 10367 webserver.cc:460] Webserver started at http://127.3.73.254:38351/ using document root <none> and password file <none>
I20260807 10:50:17.442920 10367 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:17.443010 10367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:17.443292 10367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:17.446377 10367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "270d01dc983e477fbb86e8bbbb97d4c6"
format_stamp: "Formatted at 2026-08-07 10:50:17 on dist-test-slave-x5fp"
server_key: "908a2d1aaba951e96ad87dbe20208cd4"
server_key_iv: "6e4dc86974aba96ce0e9741bcfbe6df1"
server_key_version: "encryptionkey@0"
I20260807 10:50:17.447206 10367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "270d01dc983e477fbb86e8bbbb97d4c6"
format_stamp: "Formatted at 2026-08-07 10:50:17 on dist-test-slave-x5fp"
server_key: "908a2d1aaba951e96ad87dbe20208cd4"
server_key_iv: "6e4dc86974aba96ce0e9741bcfbe6df1"
server_key_version: "encryptionkey@0"
I20260807 10:50:17.452572 10367 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 10:50:17.456301 10387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:17.457823 10367 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 10:50:17.458016 10367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "270d01dc983e477fbb86e8bbbb97d4c6"
format_stamp: "Formatted at 2026-08-07 10:50:17 on dist-test-slave-x5fp"
server_key: "908a2d1aaba951e96ad87dbe20208cd4"
server_key_iv: "6e4dc86974aba96ce0e9741bcfbe6df1"
server_key_version: "encryptionkey@0"
I20260807 10:50:17.458220 10367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:17.464252 10367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:17.465122 10367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:17.465402 10367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:17.485647 10367 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:34993
I20260807 10:50:17.485688 10461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:34993 every 8 connection(s)
I20260807 10:50:17.487193 10367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:50:17.490425 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 10367
I20260807 10:50:17.490648 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:50:17.491364 3367 external_mini_cluster.cc:1428] Setting key baa0073081837bc340f257940a0aa6fe
I20260807 10:50:17.491679 10462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:17.501940 10467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:17.494061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39714 (local address 127.3.73.254:34993)
0807 10:50:17.495245 (+ 1184us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:17.495264 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:17.495322 (+ 58us) server_negotiation.cc:407] Connection header received
0807 10:50:17.497292 (+ 1970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:17.497325 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:17.497952 (+ 627us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:17.498465 (+ 513us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:17.499673 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:17.499681 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:17.499699 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:50:17.499719 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:17.499802 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:50:17.499826 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:17.500081 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:17.500244 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 10:50:17.500574 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":373,"threads_started":1}
I20260807 10:50:17.507296 10462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 270d01dc983e477fbb86e8bbbb97d4c6: Bootstrap starting.
I20260807 10:50:17.510630 10462 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 270d01dc983e477fbb86e8bbbb97d4c6: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:17.511835 10462 log.cc:824] T 00000000000000000000000000000000 P 270d01dc983e477fbb86e8bbbb97d4c6: Log is configured to *not* fsync() on all Append() calls
I20260807 10:50:17.515424 10462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 270d01dc983e477fbb86e8bbbb97d4c6: No bootstrap required, opened a new log
I20260807 10:50:17.523882 10462 raft_consensus.cc:348] T 00000000000000000000000000000000 P 270d01dc983e477fbb86e8bbbb97d4c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "270d01dc983e477fbb86e8bbbb97d4c6" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 34993 } }
I20260807 10:50:17.524348 10462 raft_consensus.cc:374] T 00000000000000000000000000000000 P 270d01dc983e477fbb86e8bbbb97d4c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:17.524501 10462 raft_consensus.cc:729] T 00000000000000000000000000000000 P 270d01dc983e477fbb86e8bbbb97d4c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 270d01dc983e477fbb86e8bbbb97d4c6, State: Initialized, Role: FOLLOWER
I20260807 10:50:17.525240 10462 consensus_queue.cc:260] T 00000000000000000000000000000000 P 270d01dc983e477fbb86e8bbbb97d4c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "270d01dc983e477fbb86e8bbbb97d4c6" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 34993 } }
I20260807 10:50:17.525518 10462 raft_consensus.cc:388] T 00000000000000000000000000000000 P 270d01dc983e477fbb86e8bbbb97d4c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:50:17.525679 10462 raft_consensus.cc:482] T 00000000000000000000000000000000 P 270d01dc983e477fbb86e8bbbb97d4c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:50:17.525858 10462 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 270d01dc983e477fbb86e8bbbb97d4c6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:17.528475 10462 raft_consensus.cc:504] T 00000000000000000000000000000000 P 270d01dc983e477fbb86e8bbbb97d4c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "270d01dc983e477fbb86e8bbbb97d4c6" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 34993 } }
I20260807 10:50:17.529157 10462 leader_election.cc:302] T 00000000000000000000000000000000 P 270d01dc983e477fbb86e8bbbb97d4c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 270d01dc983e477fbb86e8bbbb97d4c6; no voters:
I20260807 10:50:17.529940 10462 leader_election.cc:288] T 00000000000000000000000000000000 P 270d01dc983e477fbb86e8bbbb97d4c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:50:17.530113 10469 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 270d01dc983e477fbb86e8bbbb97d4c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:50:17.530956 10469 raft_consensus.cc:686] T 00000000000000000000000000000000 P 270d01dc983e477fbb86e8bbbb97d4c6 [term 1 LEADER]: Becoming Leader. State: Replica: 270d01dc983e477fbb86e8bbbb97d4c6, State: Running, Role: LEADER
I20260807 10:50:17.531549 10469 consensus_queue.cc:237] T 00000000000000000000000000000000 P 270d01dc983e477fbb86e8bbbb97d4c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "270d01dc983e477fbb86e8bbbb97d4c6" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 34993 } }
I20260807 10:50:17.532357 10462 sys_catalog.cc:564] T 00000000000000000000000000000000 P 270d01dc983e477fbb86e8bbbb97d4c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:50:17.534582 10472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 270d01dc983e477fbb86e8bbbb97d4c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 270d01dc983e477fbb86e8bbbb97d4c6. Latest consensus state: current_term: 1 leader_uuid: "270d01dc983e477fbb86e8bbbb97d4c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "270d01dc983e477fbb86e8bbbb97d4c6" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 34993 } } }
I20260807 10:50:17.535101 10472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 270d01dc983e477fbb86e8bbbb97d4c6 [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:17.534708 10470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 270d01dc983e477fbb86e8bbbb97d4c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "270d01dc983e477fbb86e8bbbb97d4c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "270d01dc983e477fbb86e8bbbb97d4c6" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 34993 } } }
I20260807 10:50:17.535673 10470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 270d01dc983e477fbb86e8bbbb97d4c6 [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:17.542672 10481 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:50:17.548811 10481 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:50:17.559414 10481 catalog_manager.cc:1347] Generated new cluster ID: ca14aa42373d45a49c76149f2f7f41c4
I20260807 10:50:17.559566 10481 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:50:17.580967 10481 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:50:17.581423 10481 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:50:17.594873 10481 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 270d01dc983e477fbb86e8bbbb97d4c6: Generated new TSK 0
I20260807 10:50:17.595937 10481 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 10:50:17.607988 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.193:0
--local_ip_for_outbound_sockets=127.3.73.193
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:34993
--builtin_ntp_servers=127.3.73.212:40099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 10:50:17.995030 10501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:17.995357 10501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:17.995452 10501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:18.003320 10501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:18.003440 10501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:18.003556 10501 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.193
I20260807 10:50:18.014262 10501 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:40099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:34993
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.193
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:18.015638 10501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:18.017627 10501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:18.032990 10515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:18.033097 10518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:18.033078 10516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:18.034699 10501 server_base.cc:1034] running on GCE node
I20260807 10:50:18.035506 10501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:18.036819 10501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:18.038132 10501 hybrid_clock.cc:648] HybridClock initialized: now 1786099818038081 us; error 61 us; skew 500 ppm
I20260807 10:50:18.038542 10501 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 10:50:18.041711 10501 webserver.cc:460] Webserver started at http://127.3.73.193:44153/ using document root <none> and password file <none>
I20260807 10:50:18.042742 10501 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:18.042853 10501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:18.043236 10501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:18.046857 10501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-0/data/instance:
uuid: "dcb26328ff68410784f7f197d116f779"
format_stamp: "Formatted at 2026-08-07 10:50:18 on dist-test-slave-x5fp"
server_key: "02c01f1746a8e003187c6b760f490e92"
server_key_iv: "831261eceac0200e5a223b0750c4c504"
server_key_version: "encryptionkey@0"
I20260807 10:50:18.047804 10501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance:
uuid: "dcb26328ff68410784f7f197d116f779"
format_stamp: "Formatted at 2026-08-07 10:50:18 on dist-test-slave-x5fp"
server_key: "02c01f1746a8e003187c6b760f490e92"
server_key_iv: "831261eceac0200e5a223b0750c4c504"
server_key_version: "encryptionkey@0"
I20260807 10:50:18.054813 10501 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260807 10:50:18.059268 10525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:18.061458 10501 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 10:50:18.061679 10501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-0/wal
uuid: "dcb26328ff68410784f7f197d116f779"
format_stamp: "Formatted at 2026-08-07 10:50:18 on dist-test-slave-x5fp"
server_key: "02c01f1746a8e003187c6b760f490e92"
server_key_iv: "831261eceac0200e5a223b0750c4c504"
server_key_version: "encryptionkey@0"
I20260807 10:50:18.061964 10501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:18.068425 10501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:18.069344 10501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:18.069593 10501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:18.070547 10501 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:50:18.072044 10501 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:50:18.072135 10501 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:18.072214 10501 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:50:18.072273 10501 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:18.109395 10501 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.193:39379
I20260807 10:50:18.109453 10684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.193:39379 every 8 connection(s)
I20260807 10:50:18.110865 10501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
I20260807 10:50:18.112103 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 10501
I20260807 10:50:18.112322 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance
I20260807 10:50:18.113102 3367 external_mini_cluster.cc:1428] Setting key 28ea353d6c82ca293256415c256324b8
I20260807 10:50:18.116353 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.194:0
--local_ip_for_outbound_sockets=127.3.73.194
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:34993
--builtin_ntp_servers=127.3.73.212:40099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 10:50:18.120609 10688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:18.113869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:60603 (local address 127.3.73.254:34993)
0807 10:50:18.114439 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:18.114444 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:18.115959 (+ 1515us) server_negotiation.cc:407] Connection header received
0807 10:50:18.117868 (+ 1909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:18.117878 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:18.117957 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:18.118092 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:18.119524 (+ 1432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:18.119532 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:18.119539 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:50:18.119564 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:18.119633 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:50:18.119653 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:18.119810 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:18.120023 (+ 213us) server_negotiation.cc:299] Negotiation successful
0807 10:50:18.120108 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":333,"threads_started":1}
I20260807 10:50:18.121388 10689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:18.114599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34993 (local address 127.3.73.193:60603)
0807 10:50:18.115774 (+ 1175us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:18.115822 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:18.117579 (+ 1757us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:18.118306 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:18.118315 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:18.118440 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:18.119182 (+ 742us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 10:50:18.119185 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:18.119226 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 10:50:18.119319 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:18.119768 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:18.119776 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:18.119782 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 10:50:18.120009 (+ 227us) client_negotiation.cc:240] Negotiation successful
0807 10:50:18.120216 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":380,"threads_started":1}
I20260807 10:50:18.123037 10686 heartbeater.cc:347] Connected to a master server at 127.3.73.254:34993
I20260807 10:50:18.123381 10686 heartbeater.cc:464] Registering TS with master...
I20260807 10:50:18.124014 10686 heartbeater.cc:511] Master 127.3.73.254:34993 requested a full tablet report, sending...
I20260807 10:50:18.126803 10415 ts_manager.cc:194] Registered new tserver with Master: dcb26328ff68410784f7f197d116f779 (127.3.73.193:39379)
W20260807 10:50:18.472993 10691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:18.473362 10691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:18.473537 10691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:18.481150 10691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:18.481326 10691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:18.481524 10691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.194
I20260807 10:50:18.494854 10691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:40099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:34993
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.194
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:18.496574 10691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:18.498817 10691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:18.514824 10697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:18.514812 10700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:18.514887 10702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:18.515825 10691 server_base.cc:1034] running on GCE node
I20260807 10:50:18.516469 10691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:18.517625 10691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:18.518886 10691 hybrid_clock.cc:648] HybridClock initialized: now 1786099818518837 us; error 40 us; skew 500 ppm
I20260807 10:50:18.519219 10691 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 10:50:18.521745 10691 webserver.cc:460] Webserver started at http://127.3.73.194:37663/ using document root <none> and password file <none>
I20260807 10:50:18.522446 10691 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:18.522539 10691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:18.522825 10691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:18.526181 10691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-1/data/instance:
uuid: "bf2c217e80c04517a40fac65016165da"
format_stamp: "Formatted at 2026-08-07 10:50:18 on dist-test-slave-x5fp"
server_key: "0b15be73a5d12da3063ecaa4b5576805"
server_key_iv: "1ddee52bac6c080668a65f22a9e819a4"
server_key_version: "encryptionkey@0"
I20260807 10:50:18.527145 10691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance:
uuid: "bf2c217e80c04517a40fac65016165da"
format_stamp: "Formatted at 2026-08-07 10:50:18 on dist-test-slave-x5fp"
server_key: "0b15be73a5d12da3063ecaa4b5576805"
server_key_iv: "1ddee52bac6c080668a65f22a9e819a4"
server_key_version: "encryptionkey@0"
I20260807 10:50:18.532893 10691 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260807 10:50:18.536808 10708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:18.538322 10691 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 10:50:18.538522 10691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-1/wal
uuid: "bf2c217e80c04517a40fac65016165da"
format_stamp: "Formatted at 2026-08-07 10:50:18 on dist-test-slave-x5fp"
server_key: "0b15be73a5d12da3063ecaa4b5576805"
server_key_iv: "1ddee52bac6c080668a65f22a9e819a4"
server_key_version: "encryptionkey@0"
I20260807 10:50:18.538774 10691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:18.544952 10691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:18.545795 10691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:18.546087 10691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:18.547093 10691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:50:18.548645 10691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:50:18.548853 10691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:18.548974 10691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:50:18.549101 10691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:18.585919 10691 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.194:39713
I20260807 10:50:18.586031 10857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.194:39713 every 8 connection(s)
I20260807 10:50:18.587285 10691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
I20260807 10:50:18.596122 10688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:18.590414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:42613 (local address 127.3.73.254:34993)
0807 10:50:18.590633 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:18.590638 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:18.592048 (+ 1410us) server_negotiation.cc:407] Connection header received
0807 10:50:18.593933 (+ 1885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:18.593938 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:18.594000 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:18.594119 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:18.595486 (+ 1367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:18.595491 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:18.595494 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:50:18.595511 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:18.595546 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:50:18.595558 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:18.595677 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:18.595840 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 10:50:18.595897 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 10:50:18.596643 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 10691
I20260807 10:50:18.596879 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance
I20260807 10:50:18.597319 10862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:18.590914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34993 (local address 127.3.73.194:42613)
0807 10:50:18.591851 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:18.591895 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:18.593636 (+ 1741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:18.594331 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:18.594339 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:18.594437 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:18.595187 (+ 750us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 10:50:18.595190 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:18.595230 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 10:50:18.595321 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:18.595654 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:18.595662 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:18.595667 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 10:50:18.595887 (+ 220us) client_negotiation.cc:240] Negotiation successful
0807 10:50:18.596158 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":298,"threads_started":1}
I20260807 10:50:18.597365 3367 external_mini_cluster.cc:1428] Setting key 213f94598ffb07892c14e08e9f7d422f
I20260807 10:50:18.598944 10858 heartbeater.cc:347] Connected to a master server at 127.3.73.254:34993
I20260807 10:50:18.599282 10858 heartbeater.cc:464] Registering TS with master...
I20260807 10:50:18.600024 10858 heartbeater.cc:511] Master 127.3.73.254:34993 requested a full tablet report, sending...
I20260807 10:50:18.600929 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.195:0
--local_ip_for_outbound_sockets=127.3.73.195
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:34993
--builtin_ntp_servers=127.3.73.212:40099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 10:50:18.601802 10414 ts_manager.cc:194] Registered new tserver with Master: bf2c217e80c04517a40fac65016165da (127.3.73.194:39713)
W20260807 10:50:18.961355 10863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:18.961817 10863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:18.961995 10863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:18.969640 10863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:18.969791 10863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:18.969990 10863 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.195
I20260807 10:50:18.979290 10863 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:40099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:34993
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.195
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:18.980758 10863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:18.982781 10863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:18.996351 10873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:18.996398 10870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:18.996629 10869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:18.997707 10863 server_base.cc:1034] running on GCE node
I20260807 10:50:18.998450 10863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:18.999456 10863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:19.000671 10863 hybrid_clock.cc:648] HybridClock initialized: now 1786099819000630 us; error 32 us; skew 500 ppm
I20260807 10:50:19.001073 10863 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 10:50:19.003491 10863 webserver.cc:460] Webserver started at http://127.3.73.195:41861/ using document root <none> and password file <none>
I20260807 10:50:19.004226 10863 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:19.004349 10863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:19.004703 10863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:19.007820 10863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-2/data/instance:
uuid: "a68f26102b3e4c9cb270164532ca16f5"
format_stamp: "Formatted at 2026-08-07 10:50:19 on dist-test-slave-x5fp"
server_key: "7d7efc375d9053c4fd9f7b037a0f3f36"
server_key_iv: "678b79c05f9a45db0eeeae2d440c03f8"
server_key_version: "encryptionkey@0"
I20260807 10:50:19.008687 10863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance:
uuid: "a68f26102b3e4c9cb270164532ca16f5"
format_stamp: "Formatted at 2026-08-07 10:50:19 on dist-test-slave-x5fp"
server_key: "7d7efc375d9053c4fd9f7b037a0f3f36"
server_key_iv: "678b79c05f9a45db0eeeae2d440c03f8"
server_key_version: "encryptionkey@0"
I20260807 10:50:19.014586 10863 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 10:50:19.018285 10881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:19.019675 10863 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 10:50:19.019858 10863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-2/wal
uuid: "a68f26102b3e4c9cb270164532ca16f5"
format_stamp: "Formatted at 2026-08-07 10:50:19 on dist-test-slave-x5fp"
server_key: "7d7efc375d9053c4fd9f7b037a0f3f36"
server_key_iv: "678b79c05f9a45db0eeeae2d440c03f8"
server_key_version: "encryptionkey@0"
I20260807 10:50:19.020133 10863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:19.026088 10863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:19.026937 10863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:19.027228 10863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:19.028352 10863 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:50:19.029950 10863 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:50:19.030083 10863 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:19.030224 10863 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:50:19.030321 10863 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:19.067551 10863 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.195:40031
I20260807 10:50:19.067649 11034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.195:40031 every 8 connection(s)
I20260807 10:50:19.069015 10863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
I20260807 10:50:19.069380 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 10863
I20260807 10:50:19.069591 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance
I20260807 10:50:19.070302 3367 external_mini_cluster.cc:1428] Setting key 5754d61d77ba79eed7b551295025151c
I20260807 10:50:19.078630 10688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:19.072035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:44163 (local address 127.3.73.254:34993)
0807 10:50:19.072229 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:19.072235 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:19.074359 (+ 2124us) server_negotiation.cc:407] Connection header received
0807 10:50:19.076272 (+ 1913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:19.076276 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:19.076336 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:19.076429 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:19.077958 (+ 1529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:19.077963 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:19.077966 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:50:19.077984 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:19.078022 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:50:19.078043 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:19.078150 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:19.078338 (+ 188us) server_negotiation.cc:299] Negotiation successful
0807 10:50:19.078394 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 10:50:19.079732 11037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:19.072830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34993 (local address 127.3.73.195:44163)
0807 10:50:19.074169 (+ 1339us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:19.074230 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:19.075976 (+ 1746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:19.076680 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:19.076689 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:19.076867 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:19.077604 (+ 737us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 10:50:19.077607 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:19.077648 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 10:50:19.077746 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:19.078144 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:19.078152 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:19.078158 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 10:50:19.078418 (+ 260us) client_negotiation.cc:240] Negotiation successful
0807 10:50:19.078640 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"thread_start_us":438,"threads_started":1}
I20260807 10:50:19.081485 11035 heartbeater.cc:347] Connected to a master server at 127.3.73.254:34993
I20260807 10:50:19.081768 11035 heartbeater.cc:464] Registering TS with master...
I20260807 10:50:19.082363 11035 heartbeater.cc:511] Master 127.3.73.254:34993 requested a full tablet report, sending...
I20260807 10:50:19.083868 10414 ts_manager.cc:194] Registered new tserver with Master: a68f26102b3e4c9cb270164532ca16f5 (127.3.73.195:40031)
I20260807 10:50:19.086787 3367 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 10:50:19.094615 10688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:19.091746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39716 (local address 127.3.73.254:34993)
0807 10:50:19.091927 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:19.091932 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:19.092608 (+ 676us) server_negotiation.cc:407] Connection header received
0807 10:50:19.092805 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:19.092811 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:19.092880 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:19.092981 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:19.093974 (+ 993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:19.093978 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:19.093982 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:50:19.093997 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:19.094031 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:50:19.094043 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:19.094148 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:19.094339 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 10:50:19.094391 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 10:50:19.099227 10414 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39716:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 10:50:19.106354 10414 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 10:50:19.130581 10686 heartbeater.cc:503] Master 127.3.73.254:34993 was elected leader, sending a full tablet report...
I20260807 10:50:19.145221 11047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:19.120213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:39713 (local address 127.0.0.1:33336)
0807 10:50:19.135942 (+ 15729us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:19.136016 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:19.136238 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:19.141102 (+ 4864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:19.141110 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:19.141218 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:19.142419 (+ 1201us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 10:50:19.142425 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:19.142470 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 10:50:19.142585 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:19.143674 (+ 1089us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:19.143694 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:19.143707 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 10:50:19.144027 (+ 320us) client_negotiation.cc:240] Negotiation successful
0807 10:50:19.144176 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15402,"thread_start_us":15187,"threads_started":1}
I20260807 10:50:19.146478 11048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:19.120452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33336 (local address 127.3.73.194:39713)
0807 10:50:19.139053 (+ 18601us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:19.139066 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:19.139107 (+ 41us) server_negotiation.cc:407] Connection header received
0807 10:50:19.139196 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:19.139208 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:19.139526 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:19.139724 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:19.143188 (+ 3464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:19.143204 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:19.143227 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:50:19.143306 (+ 79us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:19.143457 (+ 151us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:50:19.143494 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:19.145603 (+ 2109us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:19.145739 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 10:50:19.145858 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18393,"thread_start_us":18249,"threads_started":1}
I20260807 10:50:19.151018 11053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:19.121470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47334 (local address 127.3.73.195:40031)
0807 10:50:19.145881 (+ 24411us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:19.145889 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:19.145926 (+ 37us) server_negotiation.cc:407] Connection header received
0807 10:50:19.145995 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:19.146004 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:19.146315 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:19.146594 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:19.148768 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:19.148779 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:19.148794 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:50:19.148814 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:19.148882 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:50:19.148917 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:19.149142 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:19.150109 (+ 967us) server_negotiation.cc:299] Negotiation successful
0807 10:50:19.150232 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24204,"thread_start_us":26512,"threads_started":1}
I20260807 10:50:19.151057 11051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:19.121464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59206 (local address 127.3.73.193:39379)
0807 10:50:19.140433 (+ 18969us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:19.140467 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:19.141941 (+ 1474us) server_negotiation.cc:407] Connection header received
0807 10:50:19.142029 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:19.142040 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:19.142471 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:19.142822 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:19.149334 (+ 6512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:19.149345 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:19.149371 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:50:19.149404 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:19.149534 (+ 130us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:50:19.149561 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:19.149898 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:19.150164 (+ 266us) server_negotiation.cc:299] Negotiation successful
0807 10:50:19.150278 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18666,"thread_start_us":19016,"threads_started":1}
I20260807 10:50:19.151854 11049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:19.120649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:40031 (local address 127.0.0.1:47334)
0807 10:50:19.139452 (+ 18803us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:19.139475 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:19.139590 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:19.146907 (+ 7317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:19.146920 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:19.146964 (+ 44us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:19.148370 (+ 1406us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 10:50:19.148374 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:19.148409 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 10:50:19.148494 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:19.149217 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:19.149224 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:19.149229 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 10:50:19.149432 (+ 203us) client_negotiation.cc:240] Negotiation successful
0807 10:50:19.149500 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18642,"thread_start_us":20251,"threads_started":1}
I20260807 10:50:19.152856 11050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:19.120768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:39379 (local address 127.0.0.1:59206)
0807 10:50:19.141499 (+ 20731us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:19.141529 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:19.141763 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:19.145144 (+ 3381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:19.145151 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:19.145190 (+ 39us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:19.148959 (+ 3769us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 10:50:19.148965 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:19.148995 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 10:50:19.149053 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:19.149970 (+ 917us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:19.149974 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:19.149980 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 10:50:19.151393 (+ 1413us) client_negotiation.cc:240] Negotiation successful
0807 10:50:19.151476 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20377,"thread_start_us":21971,"threads_started":1}
I20260807 10:50:19.151847 10776 tablet_service.cc:1467] Processing CreateTablet for tablet 982eba2941ed4a9cb632a76b3f0c9c6c (DEFAULT_TABLE table=test-table [id=361996116d994822b461fa9b840b3306]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:50:19.154506 10776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 982eba2941ed4a9cb632a76b3f0c9c6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:19.157037 10595 tablet_service.cc:1467] Processing CreateTablet for tablet 982eba2941ed4a9cb632a76b3f0c9c6c (DEFAULT_TABLE table=test-table [id=361996116d994822b461fa9b840b3306]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:50:19.157243 10953 tablet_service.cc:1467] Processing CreateTablet for tablet 982eba2941ed4a9cb632a76b3f0c9c6c (DEFAULT_TABLE table=test-table [id=361996116d994822b461fa9b840b3306]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:50:19.158718 10595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 982eba2941ed4a9cb632a76b3f0c9c6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:19.159018 10953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 982eba2941ed4a9cb632a76b3f0c9c6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:19.175542 11054 tablet_bootstrap.cc:492] T 982eba2941ed4a9cb632a76b3f0c9c6c P dcb26328ff68410784f7f197d116f779: Bootstrap starting.
I20260807 10:50:19.178460 11055 tablet_bootstrap.cc:492] T 982eba2941ed4a9cb632a76b3f0c9c6c P bf2c217e80c04517a40fac65016165da: Bootstrap starting.
I20260807 10:50:19.180567 11056 tablet_bootstrap.cc:492] T 982eba2941ed4a9cb632a76b3f0c9c6c P a68f26102b3e4c9cb270164532ca16f5: Bootstrap starting.
I20260807 10:50:19.182152 11054 tablet_bootstrap.cc:654] T 982eba2941ed4a9cb632a76b3f0c9c6c P dcb26328ff68410784f7f197d116f779: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:19.183080 11055 tablet_bootstrap.cc:654] T 982eba2941ed4a9cb632a76b3f0c9c6c P bf2c217e80c04517a40fac65016165da: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:19.183857 11054 log.cc:824] T 982eba2941ed4a9cb632a76b3f0c9c6c P dcb26328ff68410784f7f197d116f779: Log is configured to *not* fsync() on all Append() calls
I20260807 10:50:19.184079 11055 log.cc:824] T 982eba2941ed4a9cb632a76b3f0c9c6c P bf2c217e80c04517a40fac65016165da: Log is configured to *not* fsync() on all Append() calls
I20260807 10:50:19.184553 11056 tablet_bootstrap.cc:654] T 982eba2941ed4a9cb632a76b3f0c9c6c P a68f26102b3e4c9cb270164532ca16f5: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:19.185954 11056 log.cc:824] T 982eba2941ed4a9cb632a76b3f0c9c6c P a68f26102b3e4c9cb270164532ca16f5: Log is configured to *not* fsync() on all Append() calls
I20260807 10:50:19.188078 11054 tablet_bootstrap.cc:492] T 982eba2941ed4a9cb632a76b3f0c9c6c P dcb26328ff68410784f7f197d116f779: No bootstrap required, opened a new log
I20260807 10:50:19.188421 11054 ts_tablet_manager.cc:1397] T 982eba2941ed4a9cb632a76b3f0c9c6c P dcb26328ff68410784f7f197d116f779: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.004s
I20260807 10:50:19.188793 11055 tablet_bootstrap.cc:492] T 982eba2941ed4a9cb632a76b3f0c9c6c P bf2c217e80c04517a40fac65016165da: No bootstrap required, opened a new log
I20260807 10:50:19.189133 11055 ts_tablet_manager.cc:1397] T 982eba2941ed4a9cb632a76b3f0c9c6c P bf2c217e80c04517a40fac65016165da: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.007s
I20260807 10:50:19.189222 11056 tablet_bootstrap.cc:492] T 982eba2941ed4a9cb632a76b3f0c9c6c P a68f26102b3e4c9cb270164532ca16f5: No bootstrap required, opened a new log
I20260807 10:50:19.189495 11056 ts_tablet_manager.cc:1397] T 982eba2941ed4a9cb632a76b3f0c9c6c P a68f26102b3e4c9cb270164532ca16f5: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260807 10:50:19.196453 11055 raft_consensus.cc:348] T 982eba2941ed4a9cb632a76b3f0c9c6c P bf2c217e80c04517a40fac65016165da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf2c217e80c04517a40fac65016165da" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 39713 } } peers { permanent_uuid: "a68f26102b3e4c9cb270164532ca16f5" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 40031 } } peers { permanent_uuid: "dcb26328ff68410784f7f197d116f779" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 39379 } }
I20260807 10:50:19.197006 11055 raft_consensus.cc:374] T 982eba2941ed4a9cb632a76b3f0c9c6c P bf2c217e80c04517a40fac65016165da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:19.197117 11055 raft_consensus.cc:729] T 982eba2941ed4a9cb632a76b3f0c9c6c P bf2c217e80c04517a40fac65016165da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf2c217e80c04517a40fac65016165da, State: Initialized, Role: FOLLOWER
I20260807 10:50:19.197775 11055 consensus_queue.cc:260] T 982eba2941ed4a9cb632a76b3f0c9c6c P bf2c217e80c04517a40fac65016165da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf2c217e80c04517a40fac65016165da" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 39713 } } peers { permanent_uuid: "a68f26102b3e4c9cb270164532ca16f5" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 40031 } } peers { permanent_uuid: "dcb26328ff68410784f7f197d116f779" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 39379 } }
I20260807 10:50:19.199028 10858 heartbeater.cc:503] Master 127.3.73.254:34993 was elected leader, sending a full tablet report...
I20260807 10:50:19.199783 11055 ts_tablet_manager.cc:1428] T 982eba2941ed4a9cb632a76b3f0c9c6c P bf2c217e80c04517a40fac65016165da: Time spent starting tablet: real 0.011s user 0.009s sys 0.001s
I20260807 10:50:19.200309 11056 raft_consensus.cc:348] T 982eba2941ed4a9cb632a76b3f0c9c6c P a68f26102b3e4c9cb270164532ca16f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf2c217e80c04517a40fac65016165da" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 39713 } } peers { permanent_uuid: "a68f26102b3e4c9cb270164532ca16f5" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 40031 } } peers { permanent_uuid: "dcb26328ff68410784f7f197d116f779" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 39379 } }
I20260807 10:50:19.200811 11056 raft_consensus.cc:374] T 982eba2941ed4a9cb632a76b3f0c9c6c P a68f26102b3e4c9cb270164532ca16f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:19.200920 11056 raft_consensus.cc:729] T 982eba2941ed4a9cb632a76b3f0c9c6c P a68f26102b3e4c9cb270164532ca16f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a68f26102b3e4c9cb270164532ca16f5, State: Initialized, Role: FOLLOWER
I20260807 10:50:19.201591 11056 consensus_queue.cc:260] T 982eba2941ed4a9cb632a76b3f0c9c6c P a68f26102b3e4c9cb270164532ca16f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf2c217e80c04517a40fac65016165da" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 39713 } } peers { permanent_uuid: "a68f26102b3e4c9cb270164532ca16f5" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 40031 } } peers { permanent_uuid: "dcb26328ff68410784f7f197d116f779" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 39379 } }
I20260807 10:50:19.202663 11054 raft_consensus.cc:348] T 982eba2941ed4a9cb632a76b3f0c9c6c P dcb26328ff68410784f7f197d116f779 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf2c217e80c04517a40fac65016165da" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 39713 } } peers { permanent_uuid: "a68f26102b3e4c9cb270164532ca16f5" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 40031 } } peers { permanent_uuid: "dcb26328ff68410784f7f197d116f779" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 39379 } }
I20260807 10:50:19.203104 11054 raft_consensus.cc:374] T 982eba2941ed4a9cb632a76b3f0c9c6c P dcb26328ff68410784f7f197d116f779 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:19.203227 11054 raft_consensus.cc:729] T 982eba2941ed4a9cb632a76b3f0c9c6c P dcb26328ff68410784f7f197d116f779 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcb26328ff68410784f7f197d116f779, State: Initialized, Role: FOLLOWER
I20260807 10:50:19.203732 11056 ts_tablet_manager.cc:1428] T 982eba2941ed4a9cb632a76b3f0c9c6c P a68f26102b3e4c9cb270164532ca16f5: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260807 10:50:19.203969 11035 heartbeater.cc:503] Master 127.3.73.254:34993 was elected leader, sending a full tablet report...
I20260807 10:50:19.203884 11054 consensus_queue.cc:260] T 982eba2941ed4a9cb632a76b3f0c9c6c P dcb26328ff68410784f7f197d116f779 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf2c217e80c04517a40fac65016165da" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 39713 } } peers { permanent_uuid: "a68f26102b3e4c9cb270164532ca16f5" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 40031 } } peers { permanent_uuid: "dcb26328ff68410784f7f197d116f779" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 39379 } }
I20260807 10:50:19.206596 11054 ts_tablet_manager.cc:1428] T 982eba2941ed4a9cb632a76b3f0c9c6c P dcb26328ff68410784f7f197d116f779: Time spent starting tablet: real 0.018s user 0.012s sys 0.003s
I20260807 10:50:19.319511 11062 raft_consensus.cc:482] T 982eba2941ed4a9cb632a76b3f0c9c6c P bf2c217e80c04517a40fac65016165da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 10:50:19.319926 11062 raft_consensus.cc:504] T 982eba2941ed4a9cb632a76b3f0c9c6c P bf2c217e80c04517a40fac65016165da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf2c217e80c04517a40fac65016165da" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 39713 } } peers { permanent_uuid: "a68f26102b3e4c9cb270164532ca16f5" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 40031 } } peers { permanent_uuid: "dcb26328ff68410784f7f197d116f779" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 39379 } }
W20260807 10:50:19.321077 11036 tablet.cc:2369] T 982eba2941ed4a9cb632a76b3f0c9c6c P a68f26102b3e4c9cb270164532ca16f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:50:19.321918 11062 leader_election.cc:288] T 982eba2941ed4a9cb632a76b3f0c9c6c P bf2c217e80c04517a40fac65016165da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a68f26102b3e4c9cb270164532ca16f5 (127.3.73.195:40031), dcb26328ff68410784f7f197d116f779 (127.3.73.193:39379)
I20260807 10:50:19.326272 11053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:19.322461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:51729 (local address 127.3.73.195:40031)
0807 10:50:19.322727 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:19.322737 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:19.323548 (+ 811us) server_negotiation.cc:407] Connection header received
0807 10:50:19.323795 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:19.323802 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:19.323870 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:19.323989 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:19.325576 (+ 1587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:19.325583 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:19.325587 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:50:19.325607 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:19.325650 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:50:19.325663 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:19.325803 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:19.325889 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 10:50:19.325962 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 10:50:19.326490 11051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:19.322612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:48679 (local address 127.3.73.193:39379)
0807 10:50:19.322917 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:19.322927 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:19.323686 (+ 759us) server_negotiation.cc:407] Connection header received
0807 10:50:19.323871 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:19.323877 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:19.323948 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:19.324081 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:19.325794 (+ 1713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:19.325802 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:19.325807 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:50:19.325828 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:19.325871 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:50:19.325883 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:19.326042 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:19.326126 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 10:50:19.326197 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260807 10:50:19.326774 11068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:19.322346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:40031 (local address 127.3.73.194:51729)
0807 10:50:19.323427 (+ 1081us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:19.323444 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:19.323631 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:19.324037 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:19.324043 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:19.324067 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:19.325336 (+ 1269us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 10:50:19.325342 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:19.325364 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 10:50:19.325416 (+ 52us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:19.325772 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:19.325779 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:19.325785 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 10:50:19.325938 (+ 153us) client_negotiation.cc:240] Negotiation successful
0807 10:50:19.325993 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":870,"thread_start_us":646,"threads_started":1}
I20260807 10:50:19.327004 11066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:19.322346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:39379 (local address 127.3.73.194:48679)
0807 10:50:19.323546 (+ 1200us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:19.323567 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:19.323712 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:19.324106 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:19.324113 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:19.324144 (+ 31us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:19.325549 (+ 1405us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 10:50:19.325552 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:19.325566 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 10:50:19.325604 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:19.325999 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:19.326004 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:19.326009 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 10:50:19.326168 (+ 159us) client_negotiation.cc:240] Negotiation successful
0807 10:50:19.326220 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1049,"thread_start_us":805,"threads_started":1}
I20260807 10:50:19.327986 10978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "982eba2941ed4a9cb632a76b3f0c9c6c" candidate_uuid: "bf2c217e80c04517a40fac65016165da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a68f26102b3e4c9cb270164532ca16f5" is_pre_election: true
I20260807 10:50:19.328202 10621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "982eba2941ed4a9cb632a76b3f0c9c6c" candidate_uuid: "bf2c217e80c04517a40fac65016165da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcb26328ff68410784f7f197d116f779" is_pre_election: true
I20260807 10:50:19.328634 10978 raft_consensus.cc:2454] T 982eba2941ed4a9cb632a76b3f0c9c6c P a68f26102b3e4c9cb270164532ca16f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf2c217e80c04517a40fac65016165da in term 0.
I20260807 10:50:19.328819 10621 raft_consensus.cc:2454] T 982eba2941ed4a9cb632a76b3f0c9c6c P dcb26328ff68410784f7f197d116f779 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf2c217e80c04517a40fac65016165da in term 0.
I20260807 10:50:19.329460 10712 leader_election.cc:302] T 982eba2941ed4a9cb632a76b3f0c9c6c P bf2c217e80c04517a40fac65016165da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: a68f26102b3e4c9cb270164532ca16f5, bf2c217e80c04517a40fac65016165da, dcb26328ff68410784f7f197d116f779; no voters:
I20260807 10:50:19.329906 11062 raft_consensus.cc:2781] T 982eba2941ed4a9cb632a76b3f0c9c6c P bf2c217e80c04517a40fac65016165da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 10:50:19.330070 11062 raft_consensus.cc:482] T 982eba2941ed4a9cb632a76b3f0c9c6c P bf2c217e80c04517a40fac65016165da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 10:50:19.330183 11062 raft_consensus.cc:3037] T 982eba2941ed4a9cb632a76b3f0c9c6c P bf2c217e80c04517a40fac65016165da [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:19.333155 11062 raft_consensus.cc:504] T 982eba2941ed4a9cb632a76b3f0c9c6c P bf2c217e80c04517a40fac65016165da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf2c217e80c04517a40fac65016165da" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 39713 } } peers { permanent_uuid: "a68f26102b3e4c9cb270164532ca16f5" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 40031 } } peers { permanent_uuid: "dcb26328ff68410784f7f197d116f779" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 39379 } }
I20260807 10:50:19.333918 11062 leader_election.cc:288] T 982eba2941ed4a9cb632a76b3f0c9c6c P bf2c217e80c04517a40fac65016165da [CANDIDATE]: Term 1 election: Requested vote from peers a68f26102b3e4c9cb270164532ca16f5 (127.3.73.195:40031), dcb26328ff68410784f7f197d116f779 (127.3.73.193:39379)
I20260807 10:50:19.334504 10978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "982eba2941ed4a9cb632a76b3f0c9c6c" candidate_uuid: "bf2c217e80c04517a40fac65016165da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a68f26102b3e4c9cb270164532ca16f5"
I20260807 10:50:19.334532 10621 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "982eba2941ed4a9cb632a76b3f0c9c6c" candidate_uuid: "bf2c217e80c04517a40fac65016165da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcb26328ff68410784f7f197d116f779"
I20260807 10:50:19.334744 10621 raft_consensus.cc:3037] T 982eba2941ed4a9cb632a76b3f0c9c6c P dcb26328ff68410784f7f197d116f779 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:19.334757 10978 raft_consensus.cc:3037] T 982eba2941ed4a9cb632a76b3f0c9c6c P a68f26102b3e4c9cb270164532ca16f5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:19.338212 10978 raft_consensus.cc:2454] T 982eba2941ed4a9cb632a76b3f0c9c6c P a68f26102b3e4c9cb270164532ca16f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf2c217e80c04517a40fac65016165da in term 1.
I20260807 10:50:19.338212 10621 raft_consensus.cc:2454] T 982eba2941ed4a9cb632a76b3f0c9c6c P dcb26328ff68410784f7f197d116f779 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf2c217e80c04517a40fac65016165da in term 1.
I20260807 10:50:19.338740 10712 leader_election.cc:302] T 982eba2941ed4a9cb632a76b3f0c9c6c P bf2c217e80c04517a40fac65016165da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: a68f26102b3e4c9cb270164532ca16f5, bf2c217e80c04517a40fac65016165da, dcb26328ff68410784f7f197d116f779; no voters:
I20260807 10:50:19.339139 11062 raft_consensus.cc:2781] T 982eba2941ed4a9cb632a76b3f0c9c6c P bf2c217e80c04517a40fac65016165da [term 1 FOLLOWER]: Leader election won for term 1
W20260807 10:50:19.339654 10861 tablet.cc:2369] T 982eba2941ed4a9cb632a76b3f0c9c6c P bf2c217e80c04517a40fac65016165da: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:50:19.339937 11062 raft_consensus.cc:686] T 982eba2941ed4a9cb632a76b3f0c9c6c P bf2c217e80c04517a40fac65016165da [term 1 LEADER]: Becoming Leader. State: Replica: bf2c217e80c04517a40fac65016165da, State: Running, Role: LEADER
I20260807 10:50:19.340554 11062 consensus_queue.cc:237] T 982eba2941ed4a9cb632a76b3f0c9c6c P bf2c217e80c04517a40fac65016165da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf2c217e80c04517a40fac65016165da" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 39713 } } peers { permanent_uuid: "a68f26102b3e4c9cb270164532ca16f5" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 40031 } } peers { permanent_uuid: "dcb26328ff68410784f7f197d116f779" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 39379 } }
I20260807 10:50:19.345966 10415 catalog_manager.cc:5495] T 982eba2941ed4a9cb632a76b3f0c9c6c P bf2c217e80c04517a40fac65016165da reported cstate change: term changed from 0 to 1, leader changed from <none> to bf2c217e80c04517a40fac65016165da (127.3.73.194). New cstate: current_term: 1 leader_uuid: "bf2c217e80c04517a40fac65016165da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf2c217e80c04517a40fac65016165da" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 39713 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a68f26102b3e4c9cb270164532ca16f5" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 40031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dcb26328ff68410784f7f197d116f779" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 39379 } health_report { overall_health: UNKNOWN } } }
W20260807 10:50:19.363613 10687 tablet.cc:2369] T 982eba2941ed4a9cb632a76b3f0c9c6c P dcb26328ff68410784f7f197d116f779: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:50:19.403613 11048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:19.401149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33348 (local address 127.3.73.194:39713)
0807 10:50:19.401412 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:19.401419 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:19.401444 (+ 25us) server_negotiation.cc:407] Connection header received
0807 10:50:19.401642 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:19.401647 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:19.401741 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:19.401868 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:19.402917 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:19.402930 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:19.402934 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:50:19.402950 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:19.402984 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:50:19.402996 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:19.403126 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:19.403255 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 10:50:19.403322 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 10:50:19.412791 10978 raft_consensus.cc:1261] T 982eba2941ed4a9cb632a76b3f0c9c6c P a68f26102b3e4c9cb270164532ca16f5 [term 1 FOLLOWER]: Refusing update from remote peer bf2c217e80c04517a40fac65016165da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:50:19.412791 10621 raft_consensus.cc:1261] T 982eba2941ed4a9cb632a76b3f0c9c6c P dcb26328ff68410784f7f197d116f779 [term 1 FOLLOWER]: Refusing update from remote peer bf2c217e80c04517a40fac65016165da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:50:19.413913 11062 consensus_queue.cc:1035] T 982eba2941ed4a9cb632a76b3f0c9c6c P bf2c217e80c04517a40fac65016165da [LEADER]: Connected to new peer: Peer: permanent_uuid: "dcb26328ff68410784f7f197d116f779" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 39379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:50:19.414211 11069 consensus_queue.cc:1035] T 982eba2941ed4a9cb632a76b3f0c9c6c P bf2c217e80c04517a40fac65016165da [LEADER]: Connected to new peer: Peer: permanent_uuid: "a68f26102b3e4c9cb270164532ca16f5" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 40031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:50:19.444209 10415 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39716:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:50:19.444563 10415 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39716:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:50:19.449148 10415 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 270d01dc983e477fbb86e8bbbb97d4c6: Sending DeleteTablet for 3 replicas of tablet 982eba2941ed4a9cb632a76b3f0c9c6c
I20260807 10:50:19.450531 10953 tablet_service.cc:1514] Processing DeleteTablet for tablet 982eba2941ed4a9cb632a76b3f0c9c6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:50:19 UTC) from {username='slave'} at 127.0.0.1:47334
I20260807 10:50:19.450536 10776 tablet_service.cc:1514] Processing DeleteTablet for tablet 982eba2941ed4a9cb632a76b3f0c9c6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:50:19 UTC) from {username='slave'} at 127.0.0.1:33336
I20260807 10:50:19.450644 10595 tablet_service.cc:1514] Processing DeleteTablet for tablet 982eba2941ed4a9cb632a76b3f0c9c6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:50:19 UTC) from {username='slave'} at 127.0.0.1:59206
I20260807 10:50:19.454437 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 10501
I20260807 10:50:19.463122 11094 tablet_replica.cc:330] T 982eba2941ed4a9cb632a76b3f0c9c6c P dcb26328ff68410784f7f197d116f779: stopping tablet replica
I20260807 10:50:19.464129 11051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:19.456413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59214 (local address 127.3.73.193:39379)
0807 10:50:19.456629 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:19.456637 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:19.459258 (+ 2621us) server_negotiation.cc:407] Connection header received
0807 10:50:19.460468 (+ 1210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:19.460478 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:19.460644 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:19.460771 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:19.463002 (+ 2231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:19.463010 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:19.463015 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:50:19.463039 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:19.463099 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:50:19.463119 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:19.463282 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:19.463545 (+ 263us) server_negotiation.cc:299] Negotiation successful
0807 10:50:19.463620 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 10:50:19.465023 10680 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:19.466243 11092 tablet_replica.cc:330] T 982eba2941ed4a9cb632a76b3f0c9c6c P bf2c217e80c04517a40fac65016165da: stopping tablet replica
I20260807 10:50:19.466988 11092 raft_consensus.cc:2229] T 982eba2941ed4a9cb632a76b3f0c9c6c P bf2c217e80c04517a40fac65016165da [term 1 LEADER]: Raft consensus shutting down.
I20260807 10:50:19.468029 11092 raft_consensus.cc:2258] T 982eba2941ed4a9cb632a76b3f0c9c6c P bf2c217e80c04517a40fac65016165da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:50:19.469123 11090 tablet_replica.cc:330] T 982eba2941ed4a9cb632a76b3f0c9c6c P a68f26102b3e4c9cb270164532ca16f5: stopping tablet replica
I20260807 10:50:19.469700 11090 raft_consensus.cc:2229] T 982eba2941ed4a9cb632a76b3f0c9c6c P a68f26102b3e4c9cb270164532ca16f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:50:19.470372 11090 raft_consensus.cc:2258] T 982eba2941ed4a9cb632a76b3f0c9c6c P a68f26102b3e4c9cb270164532ca16f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:50:19.472904 11090 ts_tablet_manager.cc:1905] T 982eba2941ed4a9cb632a76b3f0c9c6c P a68f26102b3e4c9cb270164532ca16f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:50:19.473218 11092 ts_tablet_manager.cc:1905] T 982eba2941ed4a9cb632a76b3f0c9c6c P bf2c217e80c04517a40fac65016165da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:50:19.478092 11090 ts_tablet_manager.cc:1918] T 982eba2941ed4a9cb632a76b3f0c9c6c P a68f26102b3e4c9cb270164532ca16f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:50:19.478327 11090 log.cc:1196] T 982eba2941ed4a9cb632a76b3f0c9c6c P a68f26102b3e4c9cb270164532ca16f5: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-2/wal/wals/982eba2941ed4a9cb632a76b3f0c9c6c
I20260807 10:50:19.479101 11090 ts_tablet_manager.cc:1939] T 982eba2941ed4a9cb632a76b3f0c9c6c P a68f26102b3e4c9cb270164532ca16f5: Deleting consensus metadata
I20260807 10:50:19.481243 10394 catalog_manager.cc:4841] TS a68f26102b3e4c9cb270164532ca16f5 (127.3.73.195:40031): tablet 982eba2941ed4a9cb632a76b3f0c9c6c (table test-table [id=361996116d994822b461fa9b840b3306]) successfully deleted
I20260807 10:50:19.482095 11092 ts_tablet_manager.cc:1918] T 982eba2941ed4a9cb632a76b3f0c9c6c P bf2c217e80c04517a40fac65016165da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:50:19.482312 11092 log.cc:1196] T 982eba2941ed4a9cb632a76b3f0c9c6c P bf2c217e80c04517a40fac65016165da: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786099767749326-3367-0/minicluster-data/ts-1/wal/wals/982eba2941ed4a9cb632a76b3f0c9c6c
I20260807 10:50:19.482977 11092 ts_tablet_manager.cc:1939] T 982eba2941ed4a9cb632a76b3f0c9c6c P bf2c217e80c04517a40fac65016165da: Deleting consensus metadata
I20260807 10:50:19.485345 10388 catalog_manager.cc:4841] TS bf2c217e80c04517a40fac65016165da (127.3.73.194:39713): tablet 982eba2941ed4a9cb632a76b3f0c9c6c (table test-table [id=361996116d994822b461fa9b840b3306]) successfully deleted
I20260807 10:50:19.463792 11094 raft_consensus.cc:2229] T 982eba2941ed4a9cb632a76b3f0c9c6c P dcb26328ff68410784f7f197d116f779 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:50:19.604272 11094 raft_consensus.cc:2258] T 982eba2941ed4a9cb632a76b3f0c9c6c P dcb26328ff68410784f7f197d116f779 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:50:19.604367 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 10501
W20260807 10:50:19.642069 10390 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.73.193:39379 (error 108)
I20260807 10:50:19.642315 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 10691
I20260807 10:50:19.644460 11050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:19.643848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:39379 (local address 127.0.0.1:59230)
0807 10:50:19.644100 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:19.644270 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.193:39379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163}
W20260807 10:50:19.644695 10390 catalog_manager.cc:4584] TS dcb26328ff68410784f7f197d116f779 (127.3.73.193:39379): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 982eba2941ed4a9cb632a76b3f0c9c6c: Network error: Client connection negotiation failed: client connection to 127.3.73.193:39379: connect: Connection refused (error 111)
I20260807 10:50:19.646031 11048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:19.643538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33350 (local address 127.3.73.194:39713)
0807 10:50:19.643775 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:19.643782 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:19.643806 (+ 24us) server_negotiation.cc:407] Connection header received
0807 10:50:19.643972 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:19.643978 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:19.644052 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:19.644167 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:19.645274 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:19.645282 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:19.645292 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:50:19.645312 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:19.645361 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:50:19.645378 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:19.645507 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:19.645657 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 10:50:19.645717 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260807 10:50:19.646896 10851 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:19.699081 11050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:19.698488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:39379 (local address 127.0.0.1:59240)
0807 10:50:19.698768 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:19.698877 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.193:39379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260807 10:50:19.700543 11050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:19.700066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:39379 (local address 127.0.0.1:59248)
0807 10:50:19.700271 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:19.700380 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.193:39379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260807 10:50:19.761641 11050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:19.760910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:39379 (local address 127.0.0.1:59264)
0807 10:50:19.761172 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:19.761430 (+ 258us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.193:39379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260807 10:50:19.762975 11050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:19.762493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:39379 (local address 127.0.0.1:59276)
0807 10:50:19.762724 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:19.762818 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.193:39379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260807 10:50:19.766183 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 10691
I20260807 10:50:19.800014 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 10863
I20260807 10:50:19.803659 11053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:19.801035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47348 (local address 127.3.73.195:40031)
0807 10:50:19.801222 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:19.801227 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:19.801248 (+ 21us) server_negotiation.cc:407] Connection header received
0807 10:50:19.801414 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:19.801418 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:19.801518 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:19.801617 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:19.802958 (+ 1341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:19.802964 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:19.802967 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:50:19.802984 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:19.803040 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:50:19.803055 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:19.803196 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:19.803360 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 10:50:19.803416 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 10:50:19.804359 11030 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:19.843765 11050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:19.843120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:39379 (local address 127.0.0.1:59284)
0807 10:50:19.843410 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:19.843537 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.193:39379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260807 10:50:19.845022 11050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:19.844490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:39379 (local address 127.0.0.1:59294)
0807 10:50:19.844765 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:19.844860 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.193:39379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260807 10:50:19.896591 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 10863
I20260807 10:50:19.924126 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 10367
I20260807 10:50:19.925629 10455 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:20.043295 11050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:20.042564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:39379 (local address 127.0.0.1:59306)
0807 10:50:20.042812 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:20.042942 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.193:39379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260807 10:50:20.043430 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 10367
2026-08-07T10:50:20Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (3089 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 10:50:20.098815 3367 mini_oidc.cc:152] Starting JWKS server
I20260807 10:50:20.102245 3367 webserver.cc:460] Webserver started at https://127.0.0.1:44325/ using document root <none> and password file <none>
I20260807 10:50:20.102671 3367 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 10:50:20.103626 3367 webserver.cc:460] Webserver started at http://127.0.0.1:43203/ using document root <none> and password file <none>
2026-08-07T10:50:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:50:20Z Disabled control of system clock
I20260807 10:50:20.144675 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:36161
--webserver_interface=127.3.73.254
--webserver_port=0
--builtin_ntp_servers=127.3.73.212:40639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:36161
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44325/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786099767749326-3367-0/testchainca.cert with env {}
W20260807 10:50:20.496526 11110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:20.496949 11110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:20.497124 11110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:20.504765 11110 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:50:20.504925 11110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:20.505010 11110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:20.505126 11110 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:50:20.505206 11110 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 10:50:20.505300 11110 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 10:50:20.505378 11110 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44325/jwks.json
I20260807 10:50:20.515668 11110 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:40639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:36161
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:36161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44325/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786099767749326-3367-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:20.517172 11110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:20.518538 11110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:20.531288 11118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:20.531288 11121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:20.531436 11119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:20.532630 11110 server_base.cc:1034] running on GCE node
I20260807 10:50:20.533646 11110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:20.535305 11110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:20.536557 11110 hybrid_clock.cc:648] HybridClock initialized: now 1786099820536500 us; error 57 us; skew 500 ppm
I20260807 10:50:20.537011 11110 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 10:50:20.544696 11110 webserver.cc:460] Webserver started at http://127.3.73.254:44099/ using document root <none> and password file <none>
I20260807 10:50:20.545598 11110 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:20.545758 11110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:20.546120 11110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:20.549459 11110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "e2d1474becab42e99a3d48eb3ee5ddda"
format_stamp: "Formatted at 2026-08-07 10:50:20 on dist-test-slave-x5fp"
server_key: "9c285746c58e514835824b6d23471ac4"
server_key_iv: "3b0236b6238284f2ad5587ea00719266"
server_key_version: "encryptionkey@0"
I20260807 10:50:20.550375 11110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "e2d1474becab42e99a3d48eb3ee5ddda"
format_stamp: "Formatted at 2026-08-07 10:50:20 on dist-test-slave-x5fp"
server_key: "9c285746c58e514835824b6d23471ac4"
server_key_iv: "3b0236b6238284f2ad5587ea00719266"
server_key_version: "encryptionkey@0"
I20260807 10:50:20.556164 11110 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 10:50:20.560048 11128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:20.561730 11110 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 10:50:20.561959 11110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "e2d1474becab42e99a3d48eb3ee5ddda"
format_stamp: "Formatted at 2026-08-07 10:50:20 on dist-test-slave-x5fp"
server_key: "9c285746c58e514835824b6d23471ac4"
server_key_iv: "3b0236b6238284f2ad5587ea00719266"
server_key_version: "encryptionkey@0"
I20260807 10:50:20.562206 11110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:20.585270 11110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:20.586238 11110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:20.586594 11110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:20.608600 11110 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:36161
I20260807 10:50:20.608623 11198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:36161 every 8 connection(s)
I20260807 10:50:20.610414 11110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:50:20.612462 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 11110
I20260807 10:50:20.612804 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:50:20.613795 3367 external_mini_cluster.cc:1428] Setting key b6027d6cefa47b621fa86147096d30ee
I20260807 10:50:20.616443 11200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:20.629205 11204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:20.616078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41194 (local address 127.3.73.254:36161)
0807 10:50:20.617205 (+ 1127us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:20.617216 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:20.617262 (+ 46us) server_negotiation.cc:407] Connection header received
0807 10:50:20.619009 (+ 1747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:20.619031 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:20.619536 (+ 505us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:20.619912 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:20.620764 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:20.622150 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:20.624249 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:20.625028 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:20.625169 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:20.626665 (+ 1496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:20.626674 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:20.626688 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:50:20.627294 (+ 606us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:20.627379 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:50:20.627410 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:20.627504 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:20.627776 (+ 272us) server_negotiation.cc:299] Negotiation successful
0807 10:50:20.628142 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":420,"threads_started":1}
I20260807 10:50:20.631070 11200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2d1474becab42e99a3d48eb3ee5ddda: Bootstrap starting.
I20260807 10:50:20.633508 3367 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 10:50:20.634991 11200 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2d1474becab42e99a3d48eb3ee5ddda: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:20.636328 11200 log.cc:824] T 00000000000000000000000000000000 P e2d1474becab42e99a3d48eb3ee5ddda: Log is configured to *not* fsync() on all Append() calls
I20260807 10:50:20.639539 11200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2d1474becab42e99a3d48eb3ee5ddda: No bootstrap required, opened a new log
I20260807 10:50:20.648087 11200 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2d1474becab42e99a3d48eb3ee5ddda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2d1474becab42e99a3d48eb3ee5ddda" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 36161 } }
I20260807 10:50:20.648466 11200 raft_consensus.cc:374] T 00000000000000000000000000000000 P e2d1474becab42e99a3d48eb3ee5ddda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:20.648547 11200 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2d1474becab42e99a3d48eb3ee5ddda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2d1474becab42e99a3d48eb3ee5ddda, State: Initialized, Role: FOLLOWER
I20260807 10:50:20.649242 11200 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2d1474becab42e99a3d48eb3ee5ddda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2d1474becab42e99a3d48eb3ee5ddda" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 36161 } }
I20260807 10:50:20.649474 11200 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2d1474becab42e99a3d48eb3ee5ddda [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:50:20.649586 11200 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2d1474becab42e99a3d48eb3ee5ddda [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:50:20.649725 11200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2d1474becab42e99a3d48eb3ee5ddda [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:20.652566 11200 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2d1474becab42e99a3d48eb3ee5ddda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2d1474becab42e99a3d48eb3ee5ddda" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 36161 } }
I20260807 10:50:20.653136 11200 leader_election.cc:302] T 00000000000000000000000000000000 P e2d1474becab42e99a3d48eb3ee5ddda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2d1474becab42e99a3d48eb3ee5ddda; no voters:
I20260807 10:50:20.653717 11200 leader_election.cc:288] T 00000000000000000000000000000000 P e2d1474becab42e99a3d48eb3ee5ddda [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:50:20.653890 11206 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2d1474becab42e99a3d48eb3ee5ddda [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:50:20.654734 11206 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2d1474becab42e99a3d48eb3ee5ddda [term 1 LEADER]: Becoming Leader. State: Replica: e2d1474becab42e99a3d48eb3ee5ddda, State: Running, Role: LEADER
I20260807 10:50:20.655272 11206 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2d1474becab42e99a3d48eb3ee5ddda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2d1474becab42e99a3d48eb3ee5ddda" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 36161 } }
I20260807 10:50:20.656172 11200 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2d1474becab42e99a3d48eb3ee5ddda [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:50:20.658090 11207 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2d1474becab42e99a3d48eb3ee5ddda [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2d1474becab42e99a3d48eb3ee5ddda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2d1474becab42e99a3d48eb3ee5ddda" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 36161 } } }
I20260807 10:50:20.658506 11207 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2d1474becab42e99a3d48eb3ee5ddda [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:20.658499 11209 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2d1474becab42e99a3d48eb3ee5ddda [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2d1474becab42e99a3d48eb3ee5ddda. Latest consensus state: current_term: 1 leader_uuid: "e2d1474becab42e99a3d48eb3ee5ddda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2d1474becab42e99a3d48eb3ee5ddda" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 36161 } } }
I20260807 10:50:20.658957 11209 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2d1474becab42e99a3d48eb3ee5ddda [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:20.660777 11214 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:50:20.667026 11214 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:50:20.676424 11214 catalog_manager.cc:1347] Generated new cluster ID: 2ced10f9fb5a44558d0d5ea352067a7e
I20260807 10:50:20.676564 11214 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:50:20.691449 11214 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:50:20.693022 11214 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:50:20.711370 11214 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e2d1474becab42e99a3d48eb3ee5ddda: Generated new TSK 0
I20260807 10:50:20.712322 11214 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 10:50:20.752990 11236 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 10:50:20.753242 11236 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 10:50:20.754454 11204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:20.732214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41208 (local address 127.3.73.254:36161)
0807 10:50:20.732383 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:20.732387 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:20.733213 (+ 826us) server_negotiation.cc:407] Connection header received
0807 10:50:20.733617 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:20.733622 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:20.733704 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:20.733876 (+ 172us) server_negotiation.cc:226] Negotiated authn=JWT
0807 10:50:20.734490 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:20.735252 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:20.737019 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:20.737483 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:20.737626 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:20.738217 (+ 591us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 10:50:20.753676 (+ 15459us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44325/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 10:50:20.754117 (+ 441us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41208: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44325/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":56}
W20260807 10:50:20.754602 11204 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41208: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44325/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260807 10:50:20.756244 11234 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:50:20.732240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:36161 (local address 127.0.0.1:41208)
0807 10:50:20.733072 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:20.733090 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:20.733345 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:20.733904 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:20.733909 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:20.733921 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0807 10:50:20.734296 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:20.734307 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:20.735408 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:20.735414 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:20.736854 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:20.736867 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:20.737681 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:20.737688 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:20.737851 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:20.737911 (+ 60us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 10:50:20.754038 (+ 16127us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44325/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 10:50:20.754453 (+ 415us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.3.73.254:36161: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44325/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":375,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.3.73.254:36161: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44325/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 10:50:20.758565 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 11110
I20260807 10:50:20.759935 11194 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:20.862049 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 11110
2026-08-07T10:50:20Z chronyd exiting
I20260807 10:50:20.919994 3367 test_util.cc:175] -----------------------------------------------
I20260807 10:50:20.920151 3367 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786099767749326-3367-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (826 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 10:50:20.924307 3367 mini_oidc.cc:152] Starting JWKS server
I20260807 10:50:20.926342 3367 webserver.cc:460] Webserver started at https://127.0.0.1:36309/ using document root <none> and password file <none>
I20260807 10:50:20.926542 3367 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 10:50:20.927191 3367 webserver.cc:460] Webserver started at http://127.0.0.1:44319/ using document root <none> and password file <none>
2026-08-07T10:50:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:50:20Z Disabled control of system clock
I20260807 10:50:20.969677 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:43743
--webserver_interface=127.3.73.254
--webserver_port=0
--builtin_ntp_servers=127.3.73.212:34869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:43743
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36309/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786099767749326-3367-0/testchainca.cert with env {}
W20260807 10:50:21.319933 11245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:21.320358 11245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:21.320542 11245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:21.328256 11245 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:50:21.328428 11245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:21.328528 11245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:21.328655 11245 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:50:21.328758 11245 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 10:50:21.328855 11245 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 10:50:21.328943 11245 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36309/jwks.json
I20260807 10:50:21.339581 11245 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:34869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:43743
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:43743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36309/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786099767749326-3367-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:21.341063 11245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:21.342475 11245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:21.354836 11253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:21.354873 11254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:21.354810 11256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:21.355767 11245 server_base.cc:1034] running on GCE node
I20260807 10:50:21.356706 11245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:21.358261 11245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:21.359503 11245 hybrid_clock.cc:648] HybridClock initialized: now 1786099821359469 us; error 62 us; skew 500 ppm
I20260807 10:50:21.359890 11245 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 10:50:21.362504 11245 webserver.cc:460] Webserver started at http://127.3.73.254:46201/ using document root <none> and password file <none>
I20260807 10:50:21.363272 11245 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:21.363426 11245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:21.363788 11245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:21.367012 11245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "c215125453d843fba478dd45a3b8bb48"
format_stamp: "Formatted at 2026-08-07 10:50:21 on dist-test-slave-x5fp"
server_key: "8e25095c0414906b57c24555a9a7f30d"
server_key_iv: "22ff53e825bf8237599794ea52e7cf66"
server_key_version: "encryptionkey@0"
I20260807 10:50:21.367893 11245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "c215125453d843fba478dd45a3b8bb48"
format_stamp: "Formatted at 2026-08-07 10:50:21 on dist-test-slave-x5fp"
server_key: "8e25095c0414906b57c24555a9a7f30d"
server_key_iv: "22ff53e825bf8237599794ea52e7cf66"
server_key_version: "encryptionkey@0"
I20260807 10:50:21.374673 11245 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.002s
I20260807 10:50:21.378692 11269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:21.380275 11245 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 10:50:21.380501 11245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "c215125453d843fba478dd45a3b8bb48"
format_stamp: "Formatted at 2026-08-07 10:50:21 on dist-test-slave-x5fp"
server_key: "8e25095c0414906b57c24555a9a7f30d"
server_key_iv: "22ff53e825bf8237599794ea52e7cf66"
server_key_version: "encryptionkey@0"
I20260807 10:50:21.380811 11245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:21.415802 11245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:21.416808 11245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:21.417217 11245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:21.437357 11245 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:43743
I20260807 10:50:21.437435 11333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:43743 every 8 connection(s)
I20260807 10:50:21.438835 11245 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:50:21.442669 11334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:21.448372 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 11245
I20260807 10:50:21.448581 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:50:21.449352 3367 external_mini_cluster.cc:1428] Setting key a40f23762e3eba417de86f7f838dd927
I20260807 10:50:21.454135 11334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c215125453d843fba478dd45a3b8bb48: Bootstrap starting.
I20260807 10:50:21.458058 11334 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c215125453d843fba478dd45a3b8bb48: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:21.459434 11334 log.cc:824] T 00000000000000000000000000000000 P c215125453d843fba478dd45a3b8bb48: Log is configured to *not* fsync() on all Append() calls
I20260807 10:50:21.463060 11334 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c215125453d843fba478dd45a3b8bb48: No bootstrap required, opened a new log
I20260807 10:50:21.465122 11339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:21.451594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37150 (local address 127.3.73.254:43743)
0807 10:50:21.452774 (+ 1180us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:21.452787 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:21.452835 (+ 48us) server_negotiation.cc:407] Connection header received
0807 10:50:21.454606 (+ 1771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:21.454634 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:21.455101 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:21.455470 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:21.456227 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:21.457719 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:21.459722 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:21.460507 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:21.460657 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:21.462109 (+ 1452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:21.462122 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:21.462142 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:50:21.462882 (+ 740us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:21.462989 (+ 107us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:50:21.463018 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:21.463153 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:21.463429 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 10:50:21.463794 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":440,"threads_started":1}
I20260807 10:50:21.469378 3367 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 10:50:21.472293 11334 raft_consensus.cc:348] T 00000000000000000000000000000000 P c215125453d843fba478dd45a3b8bb48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c215125453d843fba478dd45a3b8bb48" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 43743 } }
I20260807 10:50:21.472757 11334 raft_consensus.cc:374] T 00000000000000000000000000000000 P c215125453d843fba478dd45a3b8bb48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:21.472867 11334 raft_consensus.cc:729] T 00000000000000000000000000000000 P c215125453d843fba478dd45a3b8bb48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c215125453d843fba478dd45a3b8bb48, State: Initialized, Role: FOLLOWER
I20260807 10:50:21.473516 11334 consensus_queue.cc:260] T 00000000000000000000000000000000 P c215125453d843fba478dd45a3b8bb48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c215125453d843fba478dd45a3b8bb48" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 43743 } }
I20260807 10:50:21.473757 11334 raft_consensus.cc:388] T 00000000000000000000000000000000 P c215125453d843fba478dd45a3b8bb48 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:50:21.473917 11334 raft_consensus.cc:482] T 00000000000000000000000000000000 P c215125453d843fba478dd45a3b8bb48 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:50:21.474082 11334 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c215125453d843fba478dd45a3b8bb48 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:21.477061 11334 raft_consensus.cc:504] T 00000000000000000000000000000000 P c215125453d843fba478dd45a3b8bb48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c215125453d843fba478dd45a3b8bb48" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 43743 } }
I20260807 10:50:21.477662 11334 leader_election.cc:302] T 00000000000000000000000000000000 P c215125453d843fba478dd45a3b8bb48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c215125453d843fba478dd45a3b8bb48; no voters:
I20260807 10:50:21.478372 11334 leader_election.cc:288] T 00000000000000000000000000000000 P c215125453d843fba478dd45a3b8bb48 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:50:21.478575 11342 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c215125453d843fba478dd45a3b8bb48 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:50:21.479557 11342 raft_consensus.cc:686] T 00000000000000000000000000000000 P c215125453d843fba478dd45a3b8bb48 [term 1 LEADER]: Becoming Leader. State: Replica: c215125453d843fba478dd45a3b8bb48, State: Running, Role: LEADER
I20260807 10:50:21.480154 11342 consensus_queue.cc:237] T 00000000000000000000000000000000 P c215125453d843fba478dd45a3b8bb48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c215125453d843fba478dd45a3b8bb48" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 43743 } }
I20260807 10:50:21.481003 11334 sys_catalog.cc:564] T 00000000000000000000000000000000 P c215125453d843fba478dd45a3b8bb48 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:50:21.483736 11347 sys_catalog.cc:455] T 00000000000000000000000000000000 P c215125453d843fba478dd45a3b8bb48 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c215125453d843fba478dd45a3b8bb48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c215125453d843fba478dd45a3b8bb48" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 43743 } } }
I20260807 10:50:21.484326 11347 sys_catalog.cc:458] T 00000000000000000000000000000000 P c215125453d843fba478dd45a3b8bb48 [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:21.485577 11348 sys_catalog.cc:455] T 00000000000000000000000000000000 P c215125453d843fba478dd45a3b8bb48 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c215125453d843fba478dd45a3b8bb48. Latest consensus state: current_term: 1 leader_uuid: "c215125453d843fba478dd45a3b8bb48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c215125453d843fba478dd45a3b8bb48" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 43743 } } }
I20260807 10:50:21.486032 11348 sys_catalog.cc:458] T 00000000000000000000000000000000 P c215125453d843fba478dd45a3b8bb48 [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:21.489449 11354 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:50:21.495481 11354 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:50:21.506000 11354 catalog_manager.cc:1347] Generated new cluster ID: 3666b6703e4148d4aa48c3e89acdca1d
I20260807 10:50:21.506146 11354 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:50:21.522316 11354 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:50:21.523710 11354 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:50:21.530386 11354 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c215125453d843fba478dd45a3b8bb48: Generated new TSK 0
I20260807 10:50:21.531242 11354 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 10:50:21.570539 11381 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 10:50:21.570796 11381 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 10:50:21.571992 11339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:21.551335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37166 (local address 127.3.73.254:43743)
0807 10:50:21.551552 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:21.551563 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:21.551975 (+ 412us) server_negotiation.cc:407] Connection header received
0807 10:50:21.552194 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:21.552201 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:21.552307 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:21.552530 (+ 223us) server_negotiation.cc:226] Negotiated authn=JWT
0807 10:50:21.553041 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:21.554217 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:21.555601 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:21.556303 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:21.556441 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:21.557035 (+ 594us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 10:50:21.571264 (+ 14229us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36309/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 10:50:21.571694 (+ 430us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37166: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36309/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":88}
W20260807 10:50:21.572120 11380 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:50:21.551178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:43743 (local address 127.0.0.1:37166)
0807 10:50:21.551850 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:21.551871 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:21.552006 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:21.552528 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:21.552532 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:21.552541 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0807 10:50:21.552855 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:21.552866 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:21.554364 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:21.554368 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:21.555444 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:21.555455 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:21.556493 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:21.556498 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:21.556649 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:21.556683 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 10:50:21.571566 (+ 14883us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36309/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 10:50:21.571785 (+ 219us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.3.73.254:43743: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36309/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":377,"threads_started":1}
W20260807 10:50:21.572134 11339 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37166: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36309/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260807 10:50:21.573988 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 11245
I20260807 10:50:21.575289 11329 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:21.670877 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 11245
2026-08-07T10:50:21Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (810 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 10:50:21.734681 3367 mini_oidc.cc:152] Starting JWKS server
I20260807 10:50:21.736672 3367 webserver.cc:460] Webserver started at https://127.0.0.1:45593/ using document root <none> and password file <none>
I20260807 10:50:21.737031 3367 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 10:50:21.737744 3367 webserver.cc:460] Webserver started at http://127.0.0.1:37561/ using document root <none> and password file <none>
2026-08-07T10:50:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:50:21Z Disabled control of system clock
I20260807 10:50:21.779062 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:35209
--webserver_interface=127.3.73.254
--webserver_port=0
--builtin_ntp_servers=127.3.73.212:36907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:35209
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45593/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 10:50:22.143929 11392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:22.144315 11392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:22.144492 11392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:22.152007 11392 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:50:22.152164 11392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:22.152258 11392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:22.152356 11392 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:50:22.152444 11392 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 10:50:22.152537 11392 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 10:50:22.152617 11392 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45593/jwks.json
I20260807 10:50:22.162175 11392 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:36907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:35209
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:35209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45593/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:22.163708 11392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:22.165658 11392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:22.178310 11398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:22.178203 11400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:22.178262 11404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:22.179345 11392 server_base.cc:1034] running on GCE node
I20260807 10:50:22.180190 11392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:22.181715 11392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:22.182976 11392 hybrid_clock.cc:648] HybridClock initialized: now 1786099822182913 us; error 72 us; skew 500 ppm
I20260807 10:50:22.183357 11392 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 10:50:22.185732 11392 webserver.cc:460] Webserver started at http://127.3.73.254:41159/ using document root <none> and password file <none>
I20260807 10:50:22.186504 11392 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:22.186645 11392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:22.186993 11392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:22.190100 11392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "1b2a0fb8557047d2a37d226fba5bb842"
format_stamp: "Formatted at 2026-08-07 10:50:22 on dist-test-slave-x5fp"
server_key: "64dbe818096e13ed87c92596feb8ae69"
server_key_iv: "efbbeca78810210db83cf074618677bb"
server_key_version: "encryptionkey@0"
I20260807 10:50:22.190960 11392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "1b2a0fb8557047d2a37d226fba5bb842"
format_stamp: "Formatted at 2026-08-07 10:50:22 on dist-test-slave-x5fp"
server_key: "64dbe818096e13ed87c92596feb8ae69"
server_key_iv: "efbbeca78810210db83cf074618677bb"
server_key_version: "encryptionkey@0"
I20260807 10:50:22.196278 11392 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 10:50:22.199844 11410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:22.201488 11392 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 10:50:22.201719 11392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "1b2a0fb8557047d2a37d226fba5bb842"
format_stamp: "Formatted at 2026-08-07 10:50:22 on dist-test-slave-x5fp"
server_key: "64dbe818096e13ed87c92596feb8ae69"
server_key_iv: "efbbeca78810210db83cf074618677bb"
server_key_version: "encryptionkey@0"
I20260807 10:50:22.201996 11392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:22.216998 11392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:22.217847 11392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:22.218176 11392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:22.237824 11392 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:35209
I20260807 10:50:22.237849 11475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:35209 every 8 connection(s)
I20260807 10:50:22.239420 11392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:50:22.243573 11476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:22.245574 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 11392
I20260807 10:50:22.245818 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:50:22.246536 3367 external_mini_cluster.cc:1428] Setting key 4ef1c232234439c7ade30fbcd4928443
I20260807 10:50:22.257875 11476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b2a0fb8557047d2a37d226fba5bb842: Bootstrap starting.
I20260807 10:50:22.262318 11476 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b2a0fb8557047d2a37d226fba5bb842: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:22.263677 11479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:22.248777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35476 (local address 127.3.73.254:35209)
0807 10:50:22.249955 (+ 1178us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:22.249971 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:22.250027 (+ 56us) server_negotiation.cc:407] Connection header received
0807 10:50:22.252406 (+ 2379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:22.252450 (+ 44us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:22.253135 (+ 685us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:22.253634 (+ 499us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:22.254215 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:22.256320 (+ 2105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:22.257980 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:22.259028 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:22.259219 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:22.260545 (+ 1326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:22.260557 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:22.260579 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:50:22.261364 (+ 785us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:22.261495 (+ 131us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:50:22.261524 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:22.261641 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:22.261896 (+ 255us) server_negotiation.cc:299] Negotiation successful
0807 10:50:22.262257 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":358,"threads_started":1}
I20260807 10:50:22.263746 11476 log.cc:824] T 00000000000000000000000000000000 P 1b2a0fb8557047d2a37d226fba5bb842: Log is configured to *not* fsync() on all Append() calls
I20260807 10:50:22.267170 11476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b2a0fb8557047d2a37d226fba5bb842: No bootstrap required, opened a new log
I20260807 10:50:22.268111 3367 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 10:50:22.275221 11476 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b2a0fb8557047d2a37d226fba5bb842 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2a0fb8557047d2a37d226fba5bb842" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 35209 } }
I20260807 10:50:22.275615 11476 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b2a0fb8557047d2a37d226fba5bb842 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:22.275722 11476 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b2a0fb8557047d2a37d226fba5bb842 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b2a0fb8557047d2a37d226fba5bb842, State: Initialized, Role: FOLLOWER
I20260807 10:50:22.276396 11476 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b2a0fb8557047d2a37d226fba5bb842 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2a0fb8557047d2a37d226fba5bb842" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 35209 } }
I20260807 10:50:22.276676 11476 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b2a0fb8557047d2a37d226fba5bb842 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:50:22.276863 11476 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b2a0fb8557047d2a37d226fba5bb842 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:50:22.277053 11476 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b2a0fb8557047d2a37d226fba5bb842 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:22.279394 11476 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b2a0fb8557047d2a37d226fba5bb842 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2a0fb8557047d2a37d226fba5bb842" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 35209 } }
I20260807 10:50:22.280071 11476 leader_election.cc:302] T 00000000000000000000000000000000 P 1b2a0fb8557047d2a37d226fba5bb842 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b2a0fb8557047d2a37d226fba5bb842; no voters:
I20260807 10:50:22.280710 11476 leader_election.cc:288] T 00000000000000000000000000000000 P 1b2a0fb8557047d2a37d226fba5bb842 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:50:22.280978 11481 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b2a0fb8557047d2a37d226fba5bb842 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:50:22.281699 11481 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b2a0fb8557047d2a37d226fba5bb842 [term 1 LEADER]: Becoming Leader. State: Replica: 1b2a0fb8557047d2a37d226fba5bb842, State: Running, Role: LEADER
I20260807 10:50:22.282305 11481 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b2a0fb8557047d2a37d226fba5bb842 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2a0fb8557047d2a37d226fba5bb842" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 35209 } }
I20260807 10:50:22.283022 11476 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b2a0fb8557047d2a37d226fba5bb842 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:50:22.285259 11483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b2a0fb8557047d2a37d226fba5bb842 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b2a0fb8557047d2a37d226fba5bb842" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2a0fb8557047d2a37d226fba5bb842" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 35209 } } }
I20260807 10:50:22.286890 11483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b2a0fb8557047d2a37d226fba5bb842 [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:22.286257 11484 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b2a0fb8557047d2a37d226fba5bb842 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b2a0fb8557047d2a37d226fba5bb842. Latest consensus state: current_term: 1 leader_uuid: "1b2a0fb8557047d2a37d226fba5bb842" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b2a0fb8557047d2a37d226fba5bb842" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 35209 } } }
I20260807 10:50:22.288137 11484 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b2a0fb8557047d2a37d226fba5bb842 [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:22.294277 11496 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:50:22.300165 11496 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:50:22.310357 11496 catalog_manager.cc:1347] Generated new cluster ID: fb4e52a8e741415ab2454c432ad0276b
I20260807 10:50:22.310528 11496 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:50:22.325596 11496 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:50:22.327585 11496 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:50:22.337452 11496 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b2a0fb8557047d2a37d226fba5bb842: Generated new TSK 0
I20260807 10:50:22.338608 11496 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 10:50:22.365971 11479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:22.356792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35478 (local address 127.3.73.254:35209)
0807 10:50:22.357011 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:22.357018 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:22.357493 (+ 475us) server_negotiation.cc:407] Connection header received
0807 10:50:22.357747 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:22.357754 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:22.357865 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:22.358075 (+ 210us) server_negotiation.cc:226] Negotiated authn=JWT
0807 10:50:22.358536 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:22.359704 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:22.361259 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:22.361992 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:22.362136 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:22.362682 (+ 546us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 10:50:22.365299 (+ 2617us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45593/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 10:50:22.365633 (+ 334us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35478: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45593/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":70}
W20260807 10:50:22.366122 11479 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35478: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45593/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260807 10:50:22.367682 11515 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:50:22.356575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35209 (local address 127.0.0.1:35478)
0807 10:50:22.357376 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:22.357394 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:22.357538 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:22.358081 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:22.358084 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:22.358095 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0807 10:50:22.358379 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:22.358387 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:22.359882 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:22.359886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:22.361107 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:22.361120 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:22.362193 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:22.362198 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:22.362349 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:22.362384 (+ 35us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 10:50:22.365520 (+ 3136us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45593/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 10:50:22.367247 (+ 1727us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.3.73.254:35209: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45593/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":645,"mutex_wait_us":1367,"thread_start_us":483,"threads_started":1}
I20260807 10:50:22.368862 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 11392
I20260807 10:50:22.370255 11470 generic_service.cc:196] Checking for leaks (request via RPC)
E20260807 10:50:22.379789 11516 webserver.cc:533] Webserver: sslize failed:
E20260807 10:50:22.379896 11516 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 10:50:22.464775 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 11392
2026-08-07T10:50:22Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (795 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T10:50:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:50:22Z Disabled control of system clock
I20260807 10:50:22.572135 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:35067
--webserver_interface=127.3.73.254
--webserver_port=0
--builtin_ntp_servers=127.3.73.212:40031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:35067
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786099767749326-3367-0/testchainca.cert with env {}
W20260807 10:50:22.927031 11525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:22.927446 11525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:22.927618 11525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:22.935051 11525 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:50:22.935227 11525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:22.935317 11525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:22.935441 11525 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:50:22.935523 11525 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 10:50:22.935626 11525 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 10:50:22.935702 11525 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 10:50:22.946271 11525 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:40031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:35067
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:35067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786099767749326-3367-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:22.947825 11525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:22.949290 11525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:22.962460 11534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:22.962419 11535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:22.962419 11539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:22.963376 11525 server_base.cc:1034] running on GCE node
I20260807 10:50:22.964311 11525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:22.966037 11525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:22.967325 11525 hybrid_clock.cc:648] HybridClock initialized: now 1786099822967281 us; error 65 us; skew 500 ppm
I20260807 10:50:22.967748 11525 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 10:50:22.970435 11525 webserver.cc:460] Webserver started at http://127.3.73.254:44543/ using document root <none> and password file <none>
I20260807 10:50:22.971195 11525 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:22.971352 11525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:22.971709 11525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:22.974889 11525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "51039ab87c554542bf6f1a803c5b6075"
format_stamp: "Formatted at 2026-08-07 10:50:22 on dist-test-slave-x5fp"
server_key: "9be4b46efaf27e901fbf63a556d213a7"
server_key_iv: "bde24836e6e2d5b0dbe606bf2422571d"
server_key_version: "encryptionkey@0"
I20260807 10:50:22.975764 11525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "51039ab87c554542bf6f1a803c5b6075"
format_stamp: "Formatted at 2026-08-07 10:50:22 on dist-test-slave-x5fp"
server_key: "9be4b46efaf27e901fbf63a556d213a7"
server_key_iv: "bde24836e6e2d5b0dbe606bf2422571d"
server_key_version: "encryptionkey@0"
I20260807 10:50:22.981384 11525 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 10:50:22.985234 11546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:22.986693 11525 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 10:50:22.986903 11525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "51039ab87c554542bf6f1a803c5b6075"
format_stamp: "Formatted at 2026-08-07 10:50:22 on dist-test-slave-x5fp"
server_key: "9be4b46efaf27e901fbf63a556d213a7"
server_key_iv: "bde24836e6e2d5b0dbe606bf2422571d"
server_key_version: "encryptionkey@0"
I20260807 10:50:22.987151 11525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:23.012692 11525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:23.013720 11525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:23.014050 11525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:23.034144 11525 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:35067
I20260807 10:50:23.034166 11609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:35067 every 8 connection(s)
I20260807 10:50:23.035624 11525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:50:23.039667 11610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:23.041096 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 11525
I20260807 10:50:23.041301 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:50:23.042146 3367 external_mini_cluster.cc:1428] Setting key b1ce9e44d0d854ba3595498f7cf8398d
I20260807 10:50:23.053560 11610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51039ab87c554542bf6f1a803c5b6075: Bootstrap starting.
I20260807 10:50:23.057056 11613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:23.044288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38772 (local address 127.3.73.254:35067)
0807 10:50:23.045424 (+ 1136us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:23.045436 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:23.045485 (+ 49us) server_negotiation.cc:407] Connection header received
0807 10:50:23.047355 (+ 1870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:23.047380 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:23.047921 (+ 541us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:23.048328 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:23.048961 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:23.050276 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:23.051616 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:23.052304 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:23.052529 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:23.054023 (+ 1494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:23.054034 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:23.054051 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:50:23.054762 (+ 711us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:23.054878 (+ 116us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:50:23.054907 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:23.055038 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:23.055315 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 10:50:23.055708 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":430,"threads_started":1}
I20260807 10:50:23.057819 11610 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51039ab87c554542bf6f1a803c5b6075: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:23.059029 11610 log.cc:824] T 00000000000000000000000000000000 P 51039ab87c554542bf6f1a803c5b6075: Log is configured to *not* fsync() on all Append() calls
I20260807 10:50:23.061879 3367 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 10:50:23.062940 11610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51039ab87c554542bf6f1a803c5b6075: No bootstrap required, opened a new log
I20260807 10:50:23.072345 11610 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51039ab87c554542bf6f1a803c5b6075 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51039ab87c554542bf6f1a803c5b6075" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 35067 } }
I20260807 10:50:23.073040 11610 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51039ab87c554542bf6f1a803c5b6075 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:23.073211 11610 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51039ab87c554542bf6f1a803c5b6075 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51039ab87c554542bf6f1a803c5b6075, State: Initialized, Role: FOLLOWER
I20260807 10:50:23.073975 11610 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51039ab87c554542bf6f1a803c5b6075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51039ab87c554542bf6f1a803c5b6075" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 35067 } }
I20260807 10:50:23.074254 11610 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51039ab87c554542bf6f1a803c5b6075 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:50:23.074440 11610 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51039ab87c554542bf6f1a803c5b6075 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:50:23.074640 11610 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51039ab87c554542bf6f1a803c5b6075 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:23.077251 11610 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51039ab87c554542bf6f1a803c5b6075 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51039ab87c554542bf6f1a803c5b6075" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 35067 } }
I20260807 10:50:23.077957 11610 leader_election.cc:302] T 00000000000000000000000000000000 P 51039ab87c554542bf6f1a803c5b6075 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51039ab87c554542bf6f1a803c5b6075; no voters:
I20260807 10:50:23.078648 11610 leader_election.cc:288] T 00000000000000000000000000000000 P 51039ab87c554542bf6f1a803c5b6075 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:50:23.078860 11616 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51039ab87c554542bf6f1a803c5b6075 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:50:23.080003 11616 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51039ab87c554542bf6f1a803c5b6075 [term 1 LEADER]: Becoming Leader. State: Replica: 51039ab87c554542bf6f1a803c5b6075, State: Running, Role: LEADER
I20260807 10:50:23.080565 11616 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51039ab87c554542bf6f1a803c5b6075 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51039ab87c554542bf6f1a803c5b6075" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 35067 } }
I20260807 10:50:23.081007 11610 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51039ab87c554542bf6f1a803c5b6075 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:50:23.084038 11618 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51039ab87c554542bf6f1a803c5b6075 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51039ab87c554542bf6f1a803c5b6075. Latest consensus state: current_term: 1 leader_uuid: "51039ab87c554542bf6f1a803c5b6075" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51039ab87c554542bf6f1a803c5b6075" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 35067 } } }
I20260807 10:50:23.084014 11617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51039ab87c554542bf6f1a803c5b6075 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "51039ab87c554542bf6f1a803c5b6075" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51039ab87c554542bf6f1a803c5b6075" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 35067 } } }
I20260807 10:50:23.084553 11618 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51039ab87c554542bf6f1a803c5b6075 [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:23.084553 11617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51039ab87c554542bf6f1a803c5b6075 [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:23.093039 11629 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:50:23.098371 11629 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:50:23.108104 11629 catalog_manager.cc:1347] Generated new cluster ID: aaf4f2b063aa461db2366015a0f9d298
I20260807 10:50:23.108249 11629 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:50:23.122169 11629 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:50:23.123692 11629 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:50:23.132169 11629 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 51039ab87c554542bf6f1a803c5b6075: Generated new TSK 0
I20260807 10:50:23.133285 11629 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 10:50:23.218741 11613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:23.156452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38774 (local address 127.3.73.254:35067)
0807 10:50:23.156683 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:23.156692 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:23.157380 (+ 688us) server_negotiation.cc:407] Connection header received
0807 10:50:23.157762 (+ 382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:23.157772 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:23.157905 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:23.158089 (+ 184us) server_negotiation.cc:226] Negotiated authn=JWT
0807 10:50:23.158651 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:23.159408 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:23.160870 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:23.161347 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:23.161492 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:23.162080 (+ 588us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 10:50:23.217819 (+ 55739us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 10:50:23.218342 (+ 523us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38774: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":73}
W20260807 10:50:23.218932 11613 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38774: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260807 10:50:23.219100 11652 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:50:23.156412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35067 (local address 127.0.0.1:38774)
0807 10:50:23.157210 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:23.157252 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:23.157448 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:23.158137 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:23.158144 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:23.158173 (+ 29us) client_negotiation.cc:189] Negotiated authn=JWT
0807 10:50:23.158481 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:23.158491 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:23.159554 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:23.159560 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:23.160652 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:23.160665 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:23.161538 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:23.161545 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:23.161746 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:23.161783 (+ 37us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 10:50:23.218224 (+ 56441us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 10:50:23.218554 (+ 330us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.3.73.254:35067: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":414,"threads_started":1}
I20260807 10:50:23.220430 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 11525
I20260807 10:50:23.221838 11603 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:23.324000 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 11525
2026-08-07T10:50:23Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (839 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (466 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (1758 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:50:25 dist-test-slave-x5fp krb5kdc[11674](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:50:25 dist-test-slave-x5fp krb5kdc[11674](info): set up 2 sockets
Aug 07 10:50:25 dist-test-slave-x5fp krb5kdc[11674](info): commencing operation
krb5kdc: starting...
W20260807 10:50:26.912995 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.316s user 0.003s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:50:27 dist-test-slave-x5fp krb5kdc[11674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:50:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:50:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.73.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
I20260807 10:50:27.210333 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:44249
--webserver_interface=127.3.73.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:33353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:44249
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:50:27.570297 11689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:27.570638 11689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:27.570856 11689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:27.578418 11689 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:50:27.578545 11689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:27.578645 11689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:27.578766 11689 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:50:27.578832 11689 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:50:27.589200 11689 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:33353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:44249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:44249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:27.590634 11689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:27.592059 11689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:27.604578 11697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:27.604609 11700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:27.604661 11698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:27.605636 11689 server_base.cc:1034] running on GCE node
I20260807 10:50:27.606516 11689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:27.607999 11689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:27.609264 11689 hybrid_clock.cc:648] HybridClock initialized: now 1786099827609216 us; error 49 us; skew 500 ppm
I20260807 10:50:27.609643 11689 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:27 dist-test-slave-x5fp krb5kdc[11674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099827, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:27.612950 11689 init.cc:377] Logged in from keytab as kudu/127.3.73.254@KRBTEST.COM (short username kudu)
I20260807 10:50:27.614679 11689 webserver.cc:460] Webserver started at http://127.3.73.254:36357/ using document root <none> and password file <none>
I20260807 10:50:27.615419 11689 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:27.615535 11689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:27.615829 11689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:27.619138 11689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "54916cf199e74acaa530fbf18d0cd1ae"
format_stamp: "Formatted at 2026-08-07 10:50:27 on dist-test-slave-x5fp"
server_key: "dc67b4537bb0fc775178650f1fd312ef"
server_key_iv: "c86f0605c0cccd396cf8babf4bdf166b"
server_key_version: "encryptionkey@0"
I20260807 10:50:27.619951 11689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "54916cf199e74acaa530fbf18d0cd1ae"
format_stamp: "Formatted at 2026-08-07 10:50:27 on dist-test-slave-x5fp"
server_key: "dc67b4537bb0fc775178650f1fd312ef"
server_key_iv: "c86f0605c0cccd396cf8babf4bdf166b"
server_key_version: "encryptionkey@0"
I20260807 10:50:27.625804 11689 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 10:50:27.629743 11709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:27.631356 11689 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 10:50:27.631534 11689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "54916cf199e74acaa530fbf18d0cd1ae"
format_stamp: "Formatted at 2026-08-07 10:50:27 on dist-test-slave-x5fp"
server_key: "dc67b4537bb0fc775178650f1fd312ef"
server_key_iv: "c86f0605c0cccd396cf8babf4bdf166b"
server_key_version: "encryptionkey@0"
I20260807 10:50:27.631733 11689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:27.647210 11689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:27.652667 11689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:27.653033 11689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:27.674108 11689 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:44249
I20260807 10:50:27.674219 11777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:44249 every 8 connection(s)
I20260807 10:50:27.675719 11689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:50:27.680081 11778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:27.681128 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 11689
I20260807 10:50:27.681339 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:50:27.682065 3367 external_mini_cluster.cc:1428] Setting key f64d9e79519ad65d7b524f2535f938c5
I20260807 10:50:27.696336 11778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54916cf199e74acaa530fbf18d0cd1ae: Bootstrap starting.
Aug 07 10:50:27 dist-test-slave-x5fp krb5kdc[11674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:50:27.699784 11778 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54916cf199e74acaa530fbf18d0cd1ae: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:27.701318 11778 log.cc:824] T 00000000000000000000000000000000 P 54916cf199e74acaa530fbf18d0cd1ae: Log is configured to *not* fsync() on all Append() calls
I20260807 10:50:27.705304 11778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54916cf199e74acaa530fbf18d0cd1ae: No bootstrap required, opened a new log
I20260807 10:50:27.709745 11781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:27.684496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44164 (local address 127.3.73.254:44249)
0807 10:50:27.685480 (+ 984us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:27.685495 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:27.685563 (+ 68us) server_negotiation.cc:407] Connection header received
0807 10:50:27.687861 (+ 2298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:27.687894 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:27.688482 (+ 588us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:27.689029 (+ 547us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:27.690488 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:27.692586 (+ 2098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:27.694077 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:27.695226 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:27.695445 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:27.699988 (+ 4543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:27.700002 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:27.700024 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:27.700083 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:27.704448 (+ 4365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:27.705376 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:27.705385 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:27.705394 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:27.705608 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:27.706089 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:27.706096 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:27.706102 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:27.706904 (+ 802us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:27.707229 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:27.707695 (+ 466us) server_negotiation.cc:299] Negotiation successful
0807 10:50:27.708210 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":447,"threads_started":1}
I20260807 10:50:27.715467 11778 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54916cf199e74acaa530fbf18d0cd1ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54916cf199e74acaa530fbf18d0cd1ae" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 44249 } }
I20260807 10:50:27.715924 11778 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54916cf199e74acaa530fbf18d0cd1ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:27.716045 11778 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54916cf199e74acaa530fbf18d0cd1ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54916cf199e74acaa530fbf18d0cd1ae, State: Initialized, Role: FOLLOWER
I20260807 10:50:27.716751 11778 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54916cf199e74acaa530fbf18d0cd1ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54916cf199e74acaa530fbf18d0cd1ae" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 44249 } }
I20260807 10:50:27.717037 11778 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54916cf199e74acaa530fbf18d0cd1ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:50:27.717203 11778 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54916cf199e74acaa530fbf18d0cd1ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:50:27.717423 11778 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54916cf199e74acaa530fbf18d0cd1ae [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:27.719993 11778 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54916cf199e74acaa530fbf18d0cd1ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54916cf199e74acaa530fbf18d0cd1ae" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 44249 } }
I20260807 10:50:27.720510 11778 leader_election.cc:302] T 00000000000000000000000000000000 P 54916cf199e74acaa530fbf18d0cd1ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54916cf199e74acaa530fbf18d0cd1ae; no voters:
I20260807 10:50:27.721206 11778 leader_election.cc:288] T 00000000000000000000000000000000 P 54916cf199e74acaa530fbf18d0cd1ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:50:27.721405 11783 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54916cf199e74acaa530fbf18d0cd1ae [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:50:27.722175 11783 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54916cf199e74acaa530fbf18d0cd1ae [term 1 LEADER]: Becoming Leader. State: Replica: 54916cf199e74acaa530fbf18d0cd1ae, State: Running, Role: LEADER
I20260807 10:50:27.722893 11783 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54916cf199e74acaa530fbf18d0cd1ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54916cf199e74acaa530fbf18d0cd1ae" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 44249 } }
I20260807 10:50:27.723580 11778 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54916cf199e74acaa530fbf18d0cd1ae [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:50:27.726735 11786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54916cf199e74acaa530fbf18d0cd1ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54916cf199e74acaa530fbf18d0cd1ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54916cf199e74acaa530fbf18d0cd1ae" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 44249 } } }
I20260807 10:50:27.726922 11787 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54916cf199e74acaa530fbf18d0cd1ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54916cf199e74acaa530fbf18d0cd1ae. Latest consensus state: current_term: 1 leader_uuid: "54916cf199e74acaa530fbf18d0cd1ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54916cf199e74acaa530fbf18d0cd1ae" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 44249 } } }
I20260807 10:50:27.727283 11787 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54916cf199e74acaa530fbf18d0cd1ae [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:27.727295 11786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54916cf199e74acaa530fbf18d0cd1ae [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:27.730226 11794 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:50:27.736322 11794 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:50:27.748623 11794 catalog_manager.cc:1347] Generated new cluster ID: 523ec01c9ac041f2ac806e977895649a
I20260807 10:50:27.748858 11794 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:50:27.792317 11794 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:50:27.793901 11794 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:50:27.801695 11794 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54916cf199e74acaa530fbf18d0cd1ae: Generated new TSK 0
I20260807 10:50:27.802551 11794 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.73.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab.
Entry for principal kudu/127.3.73.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab.
I20260807 10:50:27.897488 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.193:0
--local_ip_for_outbound_sockets=127.3.73.193
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:44249
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:33353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:50:28.279656 11814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:28.280010 11814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:28.280189 11814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:28.288707 11814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:28.288894 11814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:28.289220 11814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.193
I20260807 10:50:28.301175 11814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:33353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:44249
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.193
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:28.302636 11814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:28.304131 11814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:28.320587 11822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:28.320537 11823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:28.320492 11825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:28.321552 11814 server_base.cc:1034] running on GCE node
I20260807 10:50:28.322317 11814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:28.323484 11814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:28.324779 11814 hybrid_clock.cc:648] HybridClock initialized: now 1786099828324647 us; error 63 us; skew 500 ppm
I20260807 10:50:28.325258 11814 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:28 dist-test-slave-x5fp krb5kdc[11674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099828, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:28.329138 11814 init.cc:377] Logged in from keytab as kudu/127.3.73.193@KRBTEST.COM (short username kudu)
I20260807 10:50:28.331148 11814 webserver.cc:460] Webserver started at http://127.3.73.193:45165/ using document root <none> and password file <none>
I20260807 10:50:28.332059 11814 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:28.332243 11814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:28.332608 11814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:28.336325 11814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-0/data/instance:
uuid: "f1129a28010f469388ef7a8347e7b6f7"
format_stamp: "Formatted at 2026-08-07 10:50:28 on dist-test-slave-x5fp"
server_key: "6ecd8d768b6c4d059252ca8d04e4d594"
server_key_iv: "89fe69b88635e3b5cbe2ef3c2b0f13e5"
server_key_version: "encryptionkey@0"
I20260807 10:50:28.337368 11814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance:
uuid: "f1129a28010f469388ef7a8347e7b6f7"
format_stamp: "Formatted at 2026-08-07 10:50:28 on dist-test-slave-x5fp"
server_key: "6ecd8d768b6c4d059252ca8d04e4d594"
server_key_iv: "89fe69b88635e3b5cbe2ef3c2b0f13e5"
server_key_version: "encryptionkey@0"
I20260807 10:50:28.343665 11814 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.003s
I20260807 10:50:28.347916 11833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:28.349588 11814 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 10:50:28.349798 11814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-0/wal
uuid: "f1129a28010f469388ef7a8347e7b6f7"
format_stamp: "Formatted at 2026-08-07 10:50:28 on dist-test-slave-x5fp"
server_key: "6ecd8d768b6c4d059252ca8d04e4d594"
server_key_iv: "89fe69b88635e3b5cbe2ef3c2b0f13e5"
server_key_version: "encryptionkey@0"
I20260807 10:50:28.350040 11814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:28.366995 11814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:28.371688 11814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:28.372020 11814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:28.373124 11814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:50:28.374619 11814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:50:28.374728 11814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:28.374888 11814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:50:28.374974 11814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:28.412299 11814 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.193:46173
I20260807 10:50:28.412434 11984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.193:46173 every 8 connection(s)
I20260807 10:50:28.414057 11814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
I20260807 10:50:28.419381 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 11814
I20260807 10:50:28.419620 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance
I20260807 10:50:28.420496 3367 external_mini_cluster.cc:1428] Setting key 44e7a75ca146672fb878e0a72eceffbe
Aug 07 10:50:28 dist-test-slave-x5fp krb5kdc[11674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099828, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:50:28.440212 11989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:28.417200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:59473 (local address 127.3.73.254:44249)
0807 10:50:28.417707 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:28.417713 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:28.418750 (+ 1037us) server_negotiation.cc:407] Connection header received
0807 10:50:28.421572 (+ 2822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:28.421578 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:28.421704 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:28.421844 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:28.423873 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:28.424838 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:28.427516 (+ 2678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:28.427976 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:28.428087 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:28.433035 (+ 4948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:28.433040 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:28.433044 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:28.433080 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:28.436511 (+ 3431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:28.437357 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:28.437364 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:28.437368 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:28.437514 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:28.438089 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:28.438096 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:28.438116 (+ 20us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:28.438516 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:28.438709 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:28.439593 (+ 884us) server_negotiation.cc:299] Negotiation successful
0807 10:50:28.439850 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":236,"threads_started":1}
I20260807 10:50:28.441380 11990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:28.417800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:44249 (local address 127.3.73.193:59473)
0807 10:50:28.418573 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:28.418622 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:28.421245 (+ 2623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:28.422110 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:28.422119 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:28.422735 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:28.423670 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:28.423705 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:28.425035 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:28.425043 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:28.427332 (+ 2289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:28.427351 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:28.428237 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:28.428246 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:28.428609 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:28.429505 (+ 896us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:28.429541 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:28.432781 (+ 3240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:28.436663 (+ 3882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:28.436673 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:28.436692 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:28.437207 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:28.437641 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:28.437646 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:28.437650 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:28.437946 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:28.438709 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:28.438718 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:28.439190 (+ 472us) client_negotiation.cc:769] Sending connection context
0807 10:50:28.439480 (+ 290us) client_negotiation.cc:240] Negotiation successful
0807 10:50:28.440042 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":301,"threads_started":1}
I20260807 10:50:28.443137 11985 heartbeater.cc:347] Connected to a master server at 127.3.73.254:44249
I20260807 10:50:28.443482 11985 heartbeater.cc:464] Registering TS with master...
I20260807 10:50:28.444295 11985 heartbeater.cc:511] Master 127.3.73.254:44249 requested a full tablet report, sending...
I20260807 10:50:28.446575 11733 ts_manager.cc:194] Registered new tserver with Master: f1129a28010f469388ef7a8347e7b6f7 (127.3.73.193:46173)
I20260807 10:50:28.448024 11733 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.193@KRBTEST.COM'} at 127.3.73.193:59473
WARNING: no policy specified for kudu/127.3.73.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab.
Entry for principal kudu/127.3.73.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab.
I20260807 10:50:28.489532 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.194:0
--local_ip_for_outbound_sockets=127.3.73.194
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:44249
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:33353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:50:28.865365 11993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:28.865690 11993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:28.865795 11993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:28.873222 11993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:28.873353 11993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:28.873473 11993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.194
I20260807 10:50:28.889029 11993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:33353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:44249
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.194
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:28.890419 11993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:28.891805 11993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:28.906627 12002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:28.906611 12001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:28.906540 12005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:28.907560 11993 server_base.cc:1034] running on GCE node
I20260807 10:50:28.908418 11993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:28.909560 11993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:28.910807 11993 hybrid_clock.cc:648] HybridClock initialized: now 1786099828910757 us; error 30 us; skew 500 ppm
I20260807 10:50:28.911217 11993 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:28 dist-test-slave-x5fp krb5kdc[11674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099828, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:28.915120 11993 init.cc:377] Logged in from keytab as kudu/127.3.73.194@KRBTEST.COM (short username kudu)
I20260807 10:50:28.916836 11993 webserver.cc:460] Webserver started at http://127.3.73.194:37199/ using document root <none> and password file <none>
I20260807 10:50:28.917613 11993 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:28.917788 11993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:28.918141 11993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:28.921487 11993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-1/data/instance:
uuid: "ef99211f4be74851b727f1b8ab948087"
format_stamp: "Formatted at 2026-08-07 10:50:28 on dist-test-slave-x5fp"
server_key: "a3244f17c0b5e7d7f82cc600c217d2f4"
server_key_iv: "beaa59c449607f3802b3c410447c1886"
server_key_version: "encryptionkey@0"
I20260807 10:50:28.922412 11993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance:
uuid: "ef99211f4be74851b727f1b8ab948087"
format_stamp: "Formatted at 2026-08-07 10:50:28 on dist-test-slave-x5fp"
server_key: "a3244f17c0b5e7d7f82cc600c217d2f4"
server_key_iv: "beaa59c449607f3802b3c410447c1886"
server_key_version: "encryptionkey@0"
I20260807 10:50:28.928481 11993 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 10:50:28.932590 12018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:28.934232 11993 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 10:50:28.934448 11993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-1/wal
uuid: "ef99211f4be74851b727f1b8ab948087"
format_stamp: "Formatted at 2026-08-07 10:50:28 on dist-test-slave-x5fp"
server_key: "a3244f17c0b5e7d7f82cc600c217d2f4"
server_key_iv: "beaa59c449607f3802b3c410447c1886"
server_key_version: "encryptionkey@0"
I20260807 10:50:28.934703 11993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:28.948908 11993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:28.953567 11993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:28.953904 11993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:28.955067 11993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:50:28.956671 11993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:50:28.956806 11993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260807 10:50:28.956966 11993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:50:28.957038 11993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:28.993424 11993 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.194:39155
I20260807 10:50:28.993479 12171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.194:39155 every 8 connection(s)
I20260807 10:50:28.994809 11993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
I20260807 10:50:29.002328 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 11993
I20260807 10:50:29.002506 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance
I20260807 10:50:29.002975 3367 external_mini_cluster.cc:1428] Setting key 890e653dea9fcdfdd206ec2ae83df8de
Aug 07 10:50:29 dist-test-slave-x5fp krb5kdc[11674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099828, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:50:29.019049 12175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:28.997669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:45775 (local address 127.3.73.254:44249)
0807 10:50:28.998142 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:28.998147 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:28.999019 (+ 872us) server_negotiation.cc:407] Connection header received
0807 10:50:29.000935 (+ 1916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:29.000946 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:29.001042 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:29.001192 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:29.002943 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:29.003783 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:29.005953 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:29.006730 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:29.006885 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:29.011241 (+ 4356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:29.011251 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:29.011258 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:29.011310 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:29.015254 (+ 3944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:29.016102 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:29.016110 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:29.016116 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:29.016277 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:29.016800 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:29.016808 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:29.016814 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:29.017310 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:29.017486 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:29.018583 (+ 1097us) server_negotiation.cc:299] Negotiation successful
0807 10:50:29.018774 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":259,"threads_started":1}
I20260807 10:50:29.020669 12176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:28.998077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:44249 (local address 127.3.73.194:45775)
0807 10:50:28.998822 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:28.998862 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:29.000587 (+ 1725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:29.001443 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:29.001451 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:29.002028 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:29.002749 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:29.002769 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:29.003951 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:29.003955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:29.005773 (+ 1818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:29.005790 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:29.006994 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:29.007000 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:29.007226 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:29.008120 (+ 894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:29.008147 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:29.010945 (+ 2798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:29.015430 (+ 4485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:29.015437 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:29.015452 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:29.015928 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:29.016397 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:29.016401 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:29.016404 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:29.016601 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:29.017509 (+ 908us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:29.017521 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:29.018064 (+ 543us) client_negotiation.cc:769] Sending connection context
0807 10:50:29.018446 (+ 382us) client_negotiation.cc:240] Negotiation successful
0807 10:50:29.019102 (+ 656us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":306,"threads_started":1}
I20260807 10:50:29.022338 12172 heartbeater.cc:347] Connected to a master server at 127.3.73.254:44249
I20260807 10:50:29.022707 12172 heartbeater.cc:464] Registering TS with master...
I20260807 10:50:29.023509 12172 heartbeater.cc:511] Master 127.3.73.254:44249 requested a full tablet report, sending...
I20260807 10:50:29.025276 11733 ts_manager.cc:194] Registered new tserver with Master: ef99211f4be74851b727f1b8ab948087 (127.3.73.194:39155)
I20260807 10:50:29.026152 11733 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.194@KRBTEST.COM'} at 127.3.73.194:45775
WARNING: no policy specified for kudu/127.3.73.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab.
Entry for principal kudu/127.3.73.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab.
I20260807 10:50:29.072821 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.195:0
--local_ip_for_outbound_sockets=127.3.73.195
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:44249
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:33353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:50:29.438911 12179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:29.439353 12179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:29.439545 12179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:29.447290 12179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:29.447482 12179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:29.447698 12179 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.195
I20260807 10:50:29.451180 11985 heartbeater.cc:503] Master 127.3.73.254:44249 was elected leader, sending a full tablet report...
I20260807 10:50:29.458920 12179 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:33353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:44249
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.195
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:29.460528 12179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:29.462098 12179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:29.477170 12190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:29.477124 12199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:29.478330 12179 server_base.cc:1034] running on GCE node
W20260807 10:50:29.479091 12187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:29.480005 12179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:29.481202 12179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:29.482465 12179 hybrid_clock.cc:648] HybridClock initialized: now 1786099829482410 us; error 44 us; skew 500 ppm
I20260807 10:50:29.482914 12179 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:29 dist-test-slave-x5fp krb5kdc[11674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099829, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:29.487377 12179 init.cc:377] Logged in from keytab as kudu/127.3.73.195@KRBTEST.COM (short username kudu)
I20260807 10:50:29.489516 12179 webserver.cc:460] Webserver started at http://127.3.73.195:45393/ using document root <none> and password file <none>
I20260807 10:50:29.490422 12179 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:29.490614 12179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:29.491001 12179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:29.494702 12179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-2/data/instance:
uuid: "d9ff642f1a0640c5a39be2d4238adb5e"
format_stamp: "Formatted at 2026-08-07 10:50:29 on dist-test-slave-x5fp"
server_key: "83bc47980ffeac7e8ce229b538e5548b"
server_key_iv: "a4c694dcd1d07c6b542a3bad6d83b2ad"
server_key_version: "encryptionkey@0"
I20260807 10:50:29.495653 12179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance:
uuid: "d9ff642f1a0640c5a39be2d4238adb5e"
format_stamp: "Formatted at 2026-08-07 10:50:29 on dist-test-slave-x5fp"
server_key: "83bc47980ffeac7e8ce229b538e5548b"
server_key_iv: "a4c694dcd1d07c6b542a3bad6d83b2ad"
server_key_version: "encryptionkey@0"
I20260807 10:50:29.501602 12179 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 10:50:29.506095 12209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:29.507474 12179 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.001s
I20260807 10:50:29.507668 12179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-2/wal
uuid: "d9ff642f1a0640c5a39be2d4238adb5e"
format_stamp: "Formatted at 2026-08-07 10:50:29 on dist-test-slave-x5fp"
server_key: "83bc47980ffeac7e8ce229b538e5548b"
server_key_iv: "a4c694dcd1d07c6b542a3bad6d83b2ad"
server_key_version: "encryptionkey@0"
I20260807 10:50:29.507870 12179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:29.522467 12179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:29.527148 12179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:29.527477 12179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:29.528519 12179 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:50:29.530005 12179 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:50:29.530112 12179 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:29.530269 12179 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:50:29.530367 12179 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:29.576531 12179 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.195:42343
I20260807 10:50:29.576562 12365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.195:42343 every 8 connection(s)
I20260807 10:50:29.578141 12179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
I20260807 10:50:29.584133 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 12179
I20260807 10:50:29.584362 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance
I20260807 10:50:29.585119 3367 external_mini_cluster.cc:1428] Setting key a9966db225d48654a6c8039f12cf7ea1
Aug 07 10:50:29 dist-test-slave-x5fp krb5kdc[11674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099829, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:50:29.607323 12369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:29.581367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:38723 (local address 127.3.73.254:44249)
0807 10:50:29.581958 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:29.581963 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:29.582827 (+ 864us) server_negotiation.cc:407] Connection header received
0807 10:50:29.584884 (+ 2057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:29.584890 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:29.584996 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:29.585252 (+ 256us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:29.587267 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:29.588594 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:29.592461 (+ 3867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:29.593200 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:29.593325 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:29.598970 (+ 5645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:29.598977 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:29.598980 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:29.599028 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:29.603616 (+ 4588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:29.604439 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:29.604444 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:29.604446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:29.604645 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:29.605233 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:29.605237 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:29.605239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:29.605587 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:29.605724 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:29.606649 (+ 925us) server_negotiation.cc:299] Negotiation successful
0807 10:50:29.606877 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":296,"threads_started":1}
I20260807 10:50:29.608420 12370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:29.581772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:44249 (local address 127.3.73.195:38723)
0807 10:50:29.582623 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:29.582663 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:29.584547 (+ 1884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:29.585538 (+ 991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:29.585547 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:29.586253 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:29.587021 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:29.587044 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:29.588846 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:29.588852 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:29.592193 (+ 3341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:29.592235 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:29.593573 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:29.593583 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:29.594193 (+ 610us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:29.595444 (+ 1251us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:29.595472 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:29.598635 (+ 3163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:29.603780 (+ 5145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:29.603788 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:29.603803 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:29.604295 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:29.604802 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:29.604806 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:29.604808 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:29.605117 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:29.605765 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:29.605771 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:29.606237 (+ 466us) client_negotiation.cc:769] Sending connection context
0807 10:50:29.606507 (+ 270us) client_negotiation.cc:240] Negotiation successful
0807 10:50:29.606985 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":398,"threads_started":1}
I20260807 10:50:29.610261 12367 heartbeater.cc:347] Connected to a master server at 127.3.73.254:44249
I20260807 10:50:29.610628 12367 heartbeater.cc:464] Registering TS with master...
I20260807 10:50:29.611898 12367 heartbeater.cc:511] Master 127.3.73.254:44249 requested a full tablet report, sending...
I20260807 10:50:29.613878 11733 ts_manager.cc:194] Registered new tserver with Master: d9ff642f1a0640c5a39be2d4238adb5e (127.3.73.195:42343)
I20260807 10:50:29.615267 11733 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.195@KRBTEST.COM'} at 127.3.73.195:38723
I20260807 10:50:29.616128 3367 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/test-admin.keytab.
Aug 07 10:50:29 dist-test-slave-x5fp krb5kdc[11674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:29.656148 3367 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 10:50:29.663184 12379 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:50:29.659708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32926 (local address 127.3.73.193:46173)
0807 10:50:29.660670 (+ 962us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:29.660678 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:29.660709 (+ 31us) server_negotiation.cc:407] Connection header received
0807 10:50:29.660966 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:29.660974 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:29.661208 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:29.661357 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:29.662594 (+ 1237us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32926: BlockingRecv error: recv got EOF from 127.0.0.1:32926 (error 108)
Metrics: {"server-negotiator.queue_time_us":791,"thread_start_us":547,"threads_started":1}
W20260807 10:50:29.663550 3367 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/krb5cc' of type FILE: Runtime error: Bad format in credentials cache (filename: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/krb5cc)
Aug 07 10:50:29 dist-test-slave-x5fp krb5kdc[11674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:29.665580 3367 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 10:50:29 dist-test-slave-x5fp krb5kdc[11674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:29.667496 3367 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 10:50:29 dist-test-slave-x5fp krb5kdc[11674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.193@KRBTEST.COM
I20260807 10:50:29.691459 12379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:29.669971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32934 (local address 127.3.73.193:46173)
0807 10:50:29.670235 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:29.670242 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:29.670558 (+ 316us) server_negotiation.cc:407] Connection header received
0807 10:50:29.670825 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:29.670830 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:29.670897 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:29.671008 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:29.672496 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:29.674060 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:29.675568 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:29.676630 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:29.676797 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:29.682307 (+ 5510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:29.682325 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:29.682348 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:29.682402 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:29.687519 (+ 5117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:29.688543 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:29.688552 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:29.688554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:29.688842 (+ 288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:29.689339 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:29.689344 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:29.689346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:29.689929 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:29.690322 (+ 393us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:29.690791 (+ 469us) server_negotiation.cc:299] Negotiation successful
0807 10:50:29.691051 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260807 10:50:29.699448 12379 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:50:29.697030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32944 (local address 127.3.73.193:46173)
0807 10:50:29.697351 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:29.697361 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:29.697742 (+ 381us) server_negotiation.cc:407] Connection header received
0807 10:50:29.697923 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:29.697927 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:29.698018 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:29.698116 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:29.699190 (+ 1074us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32944: BlockingRecv error: recv got EOF from 127.0.0.1:32944 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260807 10:50:29.699776 3367 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/krb5cc' of type FILE
Aug 07 10:50:29 dist-test-slave-x5fp krb5kdc[11674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:29.701865 3367 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 10:50:29 dist-test-slave-x5fp krb5kdc[11674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:29.703896 3367 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 10:50:29 dist-test-slave-x5fp krb5kdc[11674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.193@KRBTEST.COM
I20260807 10:50:29.726718 12379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:29.705626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32948 (local address 127.3.73.193:46173)
0807 10:50:29.705838 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:29.705843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:29.706728 (+ 885us) server_negotiation.cc:407] Connection header received
0807 10:50:29.706927 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:29.706931 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:29.707000 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:29.707094 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:29.708906 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:29.709848 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:29.711719 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:29.712632 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:29.712800 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:29.718697 (+ 5897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:29.718703 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:29.718706 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:29.718750 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:29.722692 (+ 3942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:29.723758 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:29.723763 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:29.723766 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:29.724002 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:29.724576 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:29.724581 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:29.724586 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:29.725184 (+ 598us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:29.725683 (+ 499us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:29.726112 (+ 429us) server_negotiation.cc:299] Negotiation successful
0807 10:50:29.726359 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260807 10:50:29.733141 12379 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:50:29.730423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32958 (local address 127.3.73.193:46173)
0807 10:50:29.730646 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:29.730651 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:29.730857 (+ 206us) server_negotiation.cc:407] Connection header received
0807 10:50:29.731149 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:29.731153 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:29.731236 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:29.731346 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:29.732874 (+ 1528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32958: BlockingRecv error: recv got EOF from 127.0.0.1:32958 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260807 10:50:29.733460 3367 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786099767749326-3367-0/krb5kdc/krb5cc' of type FILE
Aug 07 10:50:29 dist-test-slave-x5fp krb5kdc[11674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:29.735507 3367 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 10:50:29 dist-test-slave-x5fp krb5kdc[11674](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:29.737797 3367 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 10:50:29 dist-test-slave-x5fp krb5kdc[11674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.193@KRBTEST.COM
I20260807 10:50:29.758342 12379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:29.739904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32970 (local address 127.3.73.193:46173)
0807 10:50:29.740254 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:29.740283 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:29.740968 (+ 685us) server_negotiation.cc:407] Connection header received
0807 10:50:29.741121 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:29.741126 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:29.741195 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:29.741313 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:29.743159 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:29.744059 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:29.745901 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:29.746551 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:29.746670 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:29.751798 (+ 5128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:29.751804 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:29.751807 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:29.751852 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:29.755175 (+ 3323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:29.755978 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:29.755983 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:29.755985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:29.756155 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:29.756618 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:29.756622 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:29.756624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:29.757102 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:29.757476 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:29.757740 (+ 264us) server_negotiation.cc:299] Negotiation successful
0807 10:50:29.757986 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260807 10:50:29.760527 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 11814
I20260807 10:50:29.775055 12379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:29.761569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32980 (local address 127.3.73.193:46173)
0807 10:50:29.761791 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:29.761796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:29.762322 (+ 526us) server_negotiation.cc:407] Connection header received
0807 10:50:29.762449 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:29.762452 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:29.762517 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:29.762606 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:29.764304 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:29.765131 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:29.766562 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:29.767014 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:29.767134 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:29.769551 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:29.769556 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:29.769560 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:29.769593 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:29.772286 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:29.773107 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:29.773111 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:29.773114 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:29.773245 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:29.773666 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:29.773669 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:29.773671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:29.773980 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:29.774211 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:29.774540 (+ 329us) server_negotiation.cc:299] Negotiation successful
0807 10:50:29.774717 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 10:50:29.775838 11979 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:29.865000 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 11814
I20260807 10:50:29.895064 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 11993
Aug 07 10:50:29 dist-test-slave-x5fp krb5kdc[11674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.194@KRBTEST.COM
I20260807 10:50:29.914288 12396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:29.896265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39234 (local address 127.3.73.194:39155)
0807 10:50:29.896965 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:29.896972 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:29.897001 (+ 29us) server_negotiation.cc:407] Connection header received
0807 10:50:29.897067 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:29.897074 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:29.897292 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:29.897438 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:29.898708 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:29.899843 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:29.901540 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:29.902197 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:29.902319 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:29.906811 (+ 4492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:29.906829 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:29.906846 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:29.906895 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:29.910718 (+ 3823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:29.911769 (+ 1051us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:29.911778 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:29.911780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:29.911982 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:29.912491 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:29.912495 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:29.912497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:29.912996 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:29.913251 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:29.913626 (+ 375us) server_negotiation.cc:299] Negotiation successful
0807 10:50:29.913881 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":348,"threads_started":1}
I20260807 10:50:29.915670 12166 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:30.006851 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 11993
I20260807 10:50:30.032166 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 12179
Aug 07 10:50:30 dist-test-slave-x5fp krb5kdc[11674](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.195@KRBTEST.COM
I20260807 10:50:30.050565 12399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:30.033231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33988 (local address 127.3.73.195:42343)
0807 10:50:30.033837 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:30.033844 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:30.033877 (+ 33us) server_negotiation.cc:407] Connection header received
0807 10:50:30.033941 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:30.033947 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:30.034176 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:30.034356 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:30.035495 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:30.036569 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:30.038125 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:30.038717 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:30.038865 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:30.043587 (+ 4722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:30.043604 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:30.043621 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:30.043668 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:30.047262 (+ 3594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:30.048185 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:30.048194 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:30.048196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:30.048343 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:30.048828 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:30.048834 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:30.048839 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:30.049248 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:30.049594 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:30.049920 (+ 326us) server_negotiation.cc:299] Negotiation successful
0807 10:50:30.050225 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":251,"threads_started":1}
I20260807 10:50:30.052068 12358 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:30.143923 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 12179
I20260807 10:50:30.170672 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 11689
I20260807 10:50:30.172081 11773 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:30.282305 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 11689
2026-08-07T10:50:30Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4736 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:50:30 dist-test-slave-x5fp krb5kdc[12409](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:50:30 dist-test-slave-x5fp krb5kdc[12409](info): set up 2 sockets
Aug 07 10:50:30 dist-test-slave-x5fp krb5kdc[12409](info): commencing operation
krb5kdc: starting...
W20260807 10:50:31.631487 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.299s user 0.001s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:50:31 dist-test-slave-x5fp krb5kdc[12409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:50:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:50:31Z Disabled control of system clock
WARNING: no policy specified for oryx/127.3.73.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.73.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.73.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.254.keytab.
Entry for principal oryx/127.3.73.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.254.keytab.
I20260807 10:50:31.925354 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:41969
--webserver_interface=127.3.73.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.254.keytab
--principal=oryx/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:35907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:41969
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:50:32.287504 12429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:32.287835 12429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:32.287992 12429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:32.295569 12429 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:50:32.295691 12429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:32.295779 12429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:32.295892 12429 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:50:32.295987 12429 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:50:32.306356 12429 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:41969
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.254.keytab
--principal=oryx/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:41969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:32.307794 12429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:32.309198 12429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:32.321823 12437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:32.321887 12440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:32.321847 12438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:32.323199 12429 server_base.cc:1034] running on GCE node
I20260807 10:50:32.324038 12429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:32.325690 12429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:32.326947 12429 hybrid_clock.cc:648] HybridClock initialized: now 1786099832326872 us; error 34 us; skew 500 ppm
I20260807 10:50:32.327291 12429 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:32 dist-test-slave-x5fp krb5kdc[12409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099832, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:32.336548 12429 init.cc:377] Logged in from keytab as oryx/127.3.73.254@KRBTEST.COM (short username oryx)
I20260807 10:50:32.338356 12429 webserver.cc:460] Webserver started at http://127.3.73.254:45917/ using document root <none> and password file <none>
I20260807 10:50:32.339120 12429 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:32.339236 12429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:32.339507 12429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:32.342846 12429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "ed8559261108476a81169c92d7e720db"
format_stamp: "Formatted at 2026-08-07 10:50:32 on dist-test-slave-x5fp"
server_key: "6e6ba2b982384b6bcb6588abfd156fae"
server_key_iv: "535bca67dc608eb0f2ad34d91845ac42"
server_key_version: "encryptionkey@0"
I20260807 10:50:32.343693 12429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "ed8559261108476a81169c92d7e720db"
format_stamp: "Formatted at 2026-08-07 10:50:32 on dist-test-slave-x5fp"
server_key: "6e6ba2b982384b6bcb6588abfd156fae"
server_key_iv: "535bca67dc608eb0f2ad34d91845ac42"
server_key_version: "encryptionkey@0"
I20260807 10:50:32.349972 12429 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 10:50:32.354139 12452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:32.355841 12429 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 10:50:32.356073 12429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "ed8559261108476a81169c92d7e720db"
format_stamp: "Formatted at 2026-08-07 10:50:32 on dist-test-slave-x5fp"
server_key: "6e6ba2b982384b6bcb6588abfd156fae"
server_key_iv: "535bca67dc608eb0f2ad34d91845ac42"
server_key_version: "encryptionkey@0"
I20260807 10:50:32.356312 12429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:32.381296 12429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:32.386305 12429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:32.386644 12429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:32.407194 12429 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:41969
I20260807 10:50:32.407241 12530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:41969 every 8 connection(s)
I20260807 10:50:32.408699 12429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:50:32.412945 12531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:32.416612 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 12429
I20260807 10:50:32.416900 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:50:32.417583 3367 external_mini_cluster.cc:1428] Setting key 44418893a8126141e14fa281d73f4584
I20260807 10:50:32.427153 12531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db: Bootstrap starting.
I20260807 10:50:32.431910 12531 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:32.433161 12531 log.cc:824] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db: Log is configured to *not* fsync() on all Append() calls
Aug 07 10:50:32 dist-test-slave-x5fp krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM
I20260807 10:50:32.436648 12531 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db: No bootstrap required, opened a new log
I20260807 10:50:32.442746 12536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:32.419787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56582 (local address 127.3.73.254:41969)
0807 10:50:32.420580 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:32.420591 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:32.420633 (+ 42us) server_negotiation.cc:407] Connection header received
0807 10:50:32.422412 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:32.422437 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:32.422847 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:32.423188 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:32.425316 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:32.426722 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:32.428782 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:32.430001 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:32.430253 (+ 252us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:32.434875 (+ 4622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:32.434884 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:32.434906 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:32.434950 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:32.438256 (+ 3306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:32.439036 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:32.439043 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:32.439050 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:32.439239 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:32.439683 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:32.439688 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:32.439691 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:32.440277 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:32.440645 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:32.441166 (+ 521us) server_negotiation.cc:299] Negotiation successful
0807 10:50:32.441637 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":352,"threads_started":1}
I20260807 10:50:32.445827 12531 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } }
I20260807 10:50:32.446400 12531 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:32.446524 12531 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed8559261108476a81169c92d7e720db, State: Initialized, Role: FOLLOWER
I20260807 10:50:32.447253 12531 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } }
I20260807 10:50:32.447576 12531 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:50:32.447729 12531 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:50:32.447924 12531 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:32.450942 12531 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } }
I20260807 10:50:32.451530 12531 leader_election.cc:302] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed8559261108476a81169c92d7e720db; no voters:
I20260807 10:50:32.452307 12531 leader_election.cc:288] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:50:32.452538 12539 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:50:32.453817 12539 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [term 1 LEADER]: Becoming Leader. State: Replica: ed8559261108476a81169c92d7e720db, State: Running, Role: LEADER
I20260807 10:50:32.454453 12539 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } }
I20260807 10:50:32.454921 12531 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:50:32.457649 12540 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed8559261108476a81169c92d7e720db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } }
I20260807 10:50:32.458582 12540 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:32.458984 12543 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed8559261108476a81169c92d7e720db. Latest consensus state: current_term: 1 leader_uuid: "ed8559261108476a81169c92d7e720db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } }
I20260807 10:50:32.459386 12543 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:32.461558 12555 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:50:32.466296 12555 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:50:32.475677 12555 catalog_manager.cc:1347] Generated new cluster ID: 91f30ac1709340818f64a197b4f10202
I20260807 10:50:32.475842 12555 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:50:32.495981 12555 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:50:32.498016 12555 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:50:32.513372 12555 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db: Generated new TSK 0
I20260807 10:50:32.514385 12555 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.3.73.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.73.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.73.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.193.keytab.
Entry for principal oryx/127.3.73.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.193.keytab.
I20260807 10:50:32.597537 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.193:0
--local_ip_for_outbound_sockets=127.3.73.193
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:41969
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.193.keytab
--principal=oryx/127.3.73.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:35907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:50:32.961153 12571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:32.961548 12571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:32.961745 12571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:32.970513 12571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:32.970717 12571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:32.970927 12571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.193
I20260807 10:50:32.982061 12571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.193.keytab
--principal=oryx/127.3.73.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:41969
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.193
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:32.984257 12571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:32.986260 12571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:33.001871 12584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:33.001916 12577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:33.002737 12571 server_base.cc:1034] running on GCE node
W20260807 10:50:33.003203 12578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:33.004133 12571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:33.005353 12571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:33.006583 12571 hybrid_clock.cc:648] HybridClock initialized: now 1786099833006531 us; error 32 us; skew 500 ppm
I20260807 10:50:33.006944 12571 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:33 dist-test-slave-x5fp krb5kdc[12409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099833, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:33.010459 12571 init.cc:377] Logged in from keytab as oryx/127.3.73.193@KRBTEST.COM (short username oryx)
I20260807 10:50:33.012869 12571 webserver.cc:460] Webserver started at http://127.3.73.193:42555/ using document root <none> and password file <none>
I20260807 10:50:33.013804 12571 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:33.013952 12571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:33.014319 12571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:33.017585 12571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-0/data/instance:
uuid: "6e1163382caf4551957d5c2ad39c678b"
format_stamp: "Formatted at 2026-08-07 10:50:33 on dist-test-slave-x5fp"
server_key: "e6f9807ac59144f776a772281b938147"
server_key_iv: "adcd6c131cd778bafa45cecf0e3238c3"
server_key_version: "encryptionkey@0"
I20260807 10:50:33.018425 12571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance:
uuid: "6e1163382caf4551957d5c2ad39c678b"
format_stamp: "Formatted at 2026-08-07 10:50:33 on dist-test-slave-x5fp"
server_key: "e6f9807ac59144f776a772281b938147"
server_key_iv: "adcd6c131cd778bafa45cecf0e3238c3"
server_key_version: "encryptionkey@0"
I20260807 10:50:33.024250 12571 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 10:50:33.028204 12592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:33.029661 12571 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 10:50:33.029898 12571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-0/wal
uuid: "6e1163382caf4551957d5c2ad39c678b"
format_stamp: "Formatted at 2026-08-07 10:50:33 on dist-test-slave-x5fp"
server_key: "e6f9807ac59144f776a772281b938147"
server_key_iv: "adcd6c131cd778bafa45cecf0e3238c3"
server_key_version: "encryptionkey@0"
I20260807 10:50:33.030095 12571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:33.088784 12571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:33.094028 12571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:33.094430 12571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:33.095600 12571 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:50:33.097256 12571 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:50:33.097365 12571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:33.097527 12571 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:50:33.097606 12571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:33.134651 12571 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.193:33371
I20260807 10:50:33.134687 12747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.193:33371 every 8 connection(s)
I20260807 10:50:33.136091 12571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
I20260807 10:50:33.140676 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 12571
I20260807 10:50:33.140946 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance
I20260807 10:50:33.141870 3367 external_mini_cluster.cc:1428] Setting key ccd3aa50efbb6edd5c8d580231b9ab6d
Aug 07 10:50:33 dist-test-slave-x5fp krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099833, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM
I20260807 10:50:33.162737 12751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:33.138856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:47689 (local address 127.3.73.254:41969)
0807 10:50:33.139365 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:33.139370 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:33.140601 (+ 1231us) server_negotiation.cc:407] Connection header received
0807 10:50:33.143307 (+ 2706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:33.143313 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:33.143399 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:33.143542 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:33.145724 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:33.146492 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:33.149147 (+ 2655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:33.149673 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:33.149820 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:33.154989 (+ 5169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:33.155001 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:33.155008 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:33.155067 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:33.158307 (+ 3240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:33.159455 (+ 1148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:33.159460 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:33.159462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:33.159597 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:33.160144 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:33.160148 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:33.160150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:33.160473 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:33.160589 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:33.161836 (+ 1247us) server_negotiation.cc:299] Negotiation successful
0807 10:50:33.162168 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":267,"threads_started":1}
I20260807 10:50:33.163940 12752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:33.139404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.3.73.193:47689)
0807 10:50:33.140370 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:33.140425 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:33.142943 (+ 2518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:33.143780 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:33.143789 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:33.144598 (+ 809us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:33.145503 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:33.145533 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:33.146679 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:33.146688 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:33.148949 (+ 2261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:33.148969 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:33.149953 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:33.149963 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:33.150322 (+ 359us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:33.151225 (+ 903us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:33.151261 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:33.154516 (+ 3255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:33.158508 (+ 3992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:33.158521 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:33.158545 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:33.159278 (+ 733us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:33.159721 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:33.159730 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:33.159736 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:33.160023 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:33.160629 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:33.160642 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:33.161228 (+ 586us) client_negotiation.cc:769] Sending connection context
0807 10:50:33.161625 (+ 397us) client_negotiation.cc:240] Negotiation successful
0807 10:50:33.162291 (+ 666us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":277,"threads_started":1}
I20260807 10:50:33.166051 12748 heartbeater.cc:347] Connected to a master server at 127.3.73.254:41969
I20260807 10:50:33.166450 12748 heartbeater.cc:464] Registering TS with master...
I20260807 10:50:33.167500 12748 heartbeater.cc:511] Master 127.3.73.254:41969 requested a full tablet report, sending...
I20260807 10:50:33.170265 12476 ts_manager.cc:194] Registered new tserver with Master: 6e1163382caf4551957d5c2ad39c678b (127.3.73.193:33371)
I20260807 10:50:33.171944 12476 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.73.193@KRBTEST.COM'} at 127.3.73.193:47689
WARNING: no policy specified for oryx/127.3.73.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.73.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.73.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.194.keytab.
Entry for principal oryx/127.3.73.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.194.keytab.
I20260807 10:50:33.217379 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.194:0
--local_ip_for_outbound_sockets=127.3.73.194
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:41969
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.194.keytab
--principal=oryx/127.3.73.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:35907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:50:33.590171 12757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:33.590497 12757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:33.590595 12757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:33.598206 12757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:33.598332 12757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:33.598448 12757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.194
I20260807 10:50:33.608494 12757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.194.keytab
--principal=oryx/127.3.73.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:41969
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.194
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:33.609886 12757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:33.611303 12757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:33.625365 12769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:33.625397 12771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:33.625854 12767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:33.626250 12757 server_base.cc:1034] running on GCE node
I20260807 10:50:33.626909 12757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:33.628172 12757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:33.629444 12757 hybrid_clock.cc:648] HybridClock initialized: now 1786099833629370 us; error 63 us; skew 500 ppm
I20260807 10:50:33.629786 12757 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:33 dist-test-slave-x5fp krb5kdc[12409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099833, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:33.633167 12757 init.cc:377] Logged in from keytab as oryx/127.3.73.194@KRBTEST.COM (short username oryx)
I20260807 10:50:33.634991 12757 webserver.cc:460] Webserver started at http://127.3.73.194:35231/ using document root <none> and password file <none>
I20260807 10:50:33.635767 12757 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:33.635926 12757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:33.636255 12757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:33.639348 12757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-1/data/instance:
uuid: "29153e0ab8a84b18a1db3f02cae3aefc"
format_stamp: "Formatted at 2026-08-07 10:50:33 on dist-test-slave-x5fp"
server_key: "ff538fb60b8ab206e5d5d7fa7d223446"
server_key_iv: "5da9072f69cdc0137058a83782785c1f"
server_key_version: "encryptionkey@0"
I20260807 10:50:33.640162 12757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance:
uuid: "29153e0ab8a84b18a1db3f02cae3aefc"
format_stamp: "Formatted at 2026-08-07 10:50:33 on dist-test-slave-x5fp"
server_key: "ff538fb60b8ab206e5d5d7fa7d223446"
server_key_iv: "5da9072f69cdc0137058a83782785c1f"
server_key_version: "encryptionkey@0"
I20260807 10:50:33.645581 12757 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 10:50:33.649202 12778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:33.650625 12757 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260807 10:50:33.650820 12757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-1/wal
uuid: "29153e0ab8a84b18a1db3f02cae3aefc"
format_stamp: "Formatted at 2026-08-07 10:50:33 on dist-test-slave-x5fp"
server_key: "ff538fb60b8ab206e5d5d7fa7d223446"
server_key_iv: "5da9072f69cdc0137058a83782785c1f"
server_key_version: "encryptionkey@0"
I20260807 10:50:33.651057 12757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:33.675488 12757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:33.680017 12757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:33.680343 12757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:33.681463 12757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:50:33.682890 12757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:50:33.682998 12757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:33.683156 12757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:50:33.683230 12757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:33.720687 12757 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.194:41959
I20260807 10:50:33.720896 12936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.194:41959 every 8 connection(s)
I20260807 10:50:33.722167 12757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
I20260807 10:50:33.729117 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 12757
I20260807 10:50:33.729305 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance
I20260807 10:50:33.729806 3367 external_mini_cluster.cc:1428] Setting key d579a59c21a0982ccffffdd057081e6c
Aug 07 10:50:33 dist-test-slave-x5fp krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099833, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM
I20260807 10:50:33.747577 12940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:33.725412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:55841 (local address 127.3.73.254:41969)
0807 10:50:33.725944 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:33.725949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:33.726999 (+ 1050us) server_negotiation.cc:407] Connection header received
0807 10:50:33.729104 (+ 2105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:33.729112 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:33.729229 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:33.729382 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:33.731333 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:33.732371 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:33.734312 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:33.735168 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:33.735329 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:33.739550 (+ 4221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:33.739560 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:33.739567 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:33.739619 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:33.743750 (+ 4131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:33.744691 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:33.744709 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:33.744741 (+ 32us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:33.744915 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:33.745515 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:33.745519 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:33.745521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:33.745867 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:33.745988 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:33.747074 (+ 1086us) server_negotiation.cc:299] Negotiation successful
0807 10:50:33.747273 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":261,"threads_started":1}
I20260807 10:50:33.749212 12944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:33.725975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.3.73.194:55841)
0807 10:50:33.726800 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:33.726846 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:33.728759 (+ 1913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:33.729646 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:33.729655 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:33.730271 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:33.731114 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:33.731140 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:33.732583 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:33.732588 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:33.734145 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:33.734159 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:33.735414 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:33.735420 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:33.735631 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:33.736467 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:33.736494 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:33.739255 (+ 2761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:33.743984 (+ 4729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:33.743992 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:33.744009 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:33.744532 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:33.745071 (+ 539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:33.745079 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:33.745084 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:33.745370 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:33.746017 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:33.746028 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:33.746568 (+ 540us) client_negotiation.cc:769] Sending connection context
0807 10:50:33.746927 (+ 359us) client_negotiation.cc:240] Negotiation successful
0807 10:50:33.747572 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":309,"threads_started":1}
I20260807 10:50:33.751019 12938 heartbeater.cc:347] Connected to a master server at 127.3.73.254:41969
I20260807 10:50:33.751401 12938 heartbeater.cc:464] Registering TS with master...
I20260807 10:50:33.752213 12938 heartbeater.cc:511] Master 127.3.73.254:41969 requested a full tablet report, sending...
I20260807 10:50:33.754016 12476 ts_manager.cc:194] Registered new tserver with Master: 29153e0ab8a84b18a1db3f02cae3aefc (127.3.73.194:41959)
I20260807 10:50:33.755043 12476 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.73.194@KRBTEST.COM'} at 127.3.73.194:55841
WARNING: no policy specified for oryx/127.3.73.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.73.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.73.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.195.keytab.
Entry for principal oryx/127.3.73.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.195.keytab.
I20260807 10:50:33.801582 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.195:0
--local_ip_for_outbound_sockets=127.3.73.195
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:41969
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.195.keytab
--principal=oryx/127.3.73.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:35907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 10:50:34.176020 12748 heartbeater.cc:503] Master 127.3.73.254:41969 was elected leader, sending a full tablet report...
W20260807 10:50:34.177774 12947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:34.178165 12947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:34.178363 12947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:34.185710 12947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:34.185868 12947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:34.186049 12947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.195
I20260807 10:50:34.196707 12947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.195.keytab
--principal=oryx/127.3.73.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:41969
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.195
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:34.198253 12947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:34.199651 12947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:34.213716 12953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:34.214740 12947 server_base.cc:1034] running on GCE node
W20260807 10:50:34.215047 12956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:34.215416 12954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:34.216117 12947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:34.217222 12947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:34.218478 12947 hybrid_clock.cc:648] HybridClock initialized: now 1786099834218436 us; error 41 us; skew 500 ppm
I20260807 10:50:34.218814 12947 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:34 dist-test-slave-x5fp krb5kdc[12409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099834, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:34.222889 12947 init.cc:377] Logged in from keytab as oryx/127.3.73.195@KRBTEST.COM (short username oryx)
I20260807 10:50:34.225185 12947 webserver.cc:460] Webserver started at http://127.3.73.195:39451/ using document root <none> and password file <none>
I20260807 10:50:34.225924 12947 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:34.226037 12947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:34.226351 12947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:34.229996 12947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-2/data/instance:
uuid: "0ed44d75c8c349feb64451524c6536a6"
format_stamp: "Formatted at 2026-08-07 10:50:34 on dist-test-slave-x5fp"
server_key: "caadd4a334a22b953d97d39d12a31680"
server_key_iv: "68d804cb7af62aeccc7a0f3d7d8d67f2"
server_key_version: "encryptionkey@0"
I20260807 10:50:34.230834 12947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance:
uuid: "0ed44d75c8c349feb64451524c6536a6"
format_stamp: "Formatted at 2026-08-07 10:50:34 on dist-test-slave-x5fp"
server_key: "caadd4a334a22b953d97d39d12a31680"
server_key_iv: "68d804cb7af62aeccc7a0f3d7d8d67f2"
server_key_version: "encryptionkey@0"
I20260807 10:50:34.238247 12947 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.005s
I20260807 10:50:34.243350 12964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:34.245082 12947 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 10:50:34.245342 12947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-2/wal
uuid: "0ed44d75c8c349feb64451524c6536a6"
format_stamp: "Formatted at 2026-08-07 10:50:34 on dist-test-slave-x5fp"
server_key: "caadd4a334a22b953d97d39d12a31680"
server_key_iv: "68d804cb7af62aeccc7a0f3d7d8d67f2"
server_key_version: "encryptionkey@0"
I20260807 10:50:34.245685 12947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:34.264382 12947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:34.269359 12947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:34.269713 12947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:34.270889 12947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:50:34.272881 12947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:50:34.272991 12947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:34.273157 12947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:50:34.273248 12947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:34.314388 12947 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.195:36361
I20260807 10:50:34.314488 13123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.195:36361 every 8 connection(s)
I20260807 10:50:34.315796 12947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
I20260807 10:50:34.324113 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 12947
I20260807 10:50:34.324348 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance
I20260807 10:50:34.324986 3367 external_mini_cluster.cc:1428] Setting key e087fe891e8801bf17bdf9b738893caa
Aug 07 10:50:34 dist-test-slave-x5fp krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099834, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM
I20260807 10:50:34.348599 13130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:34.318589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:32969 (local address 127.3.73.254:41969)
0807 10:50:34.319120 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:34.319127 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:34.320385 (+ 1258us) server_negotiation.cc:407] Connection header received
0807 10:50:34.324957 (+ 4572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:34.324966 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:34.325095 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:34.325238 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:34.328272 (+ 3034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:34.329600 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:34.333984 (+ 4384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:34.334984 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:34.335105 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:34.341526 (+ 6421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:34.341532 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:34.341535 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:34.341578 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:34.345016 (+ 3438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:34.345944 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:34.345953 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:34.345956 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:34.346086 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:34.346650 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:34.346659 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:34.346661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:34.346949 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:34.347127 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:34.347929 (+ 802us) server_negotiation.cc:299] Negotiation successful
0807 10:50:34.348173 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":264,"threads_started":1}
I20260807 10:50:34.349859 13131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:34.319225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.3.73.195:32969)
0807 10:50:34.320101 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:34.320210 (+ 109us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:34.324303 (+ 4093us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:34.325572 (+ 1269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:34.325589 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:34.326745 (+ 1156us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:34.328025 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:34.328057 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:34.329848 (+ 1791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:34.329855 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:34.333735 (+ 3880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:34.333761 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:34.335367 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:34.335378 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:34.336161 (+ 783us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:34.337288 (+ 1127us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:34.337315 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:34.340987 (+ 3672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:34.345185 (+ 4198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:34.345197 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:34.345224 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:34.345792 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:34.346211 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:34.346216 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:34.346220 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:34.346512 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:34.347105 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:34.347113 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:34.347512 (+ 399us) client_negotiation.cc:769] Sending connection context
0807 10:50:34.347820 (+ 308us) client_negotiation.cc:240] Negotiation successful
0807 10:50:34.348442 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":357,"threads_started":1}
I20260807 10:50:34.351673 13124 heartbeater.cc:347] Connected to a master server at 127.3.73.254:41969
I20260807 10:50:34.352133 13124 heartbeater.cc:464] Registering TS with master...
I20260807 10:50:34.353199 13124 heartbeater.cc:511] Master 127.3.73.254:41969 requested a full tablet report, sending...
I20260807 10:50:34.355402 12476 ts_manager.cc:194] Registered new tserver with Master: 0ed44d75c8c349feb64451524c6536a6 (127.3.73.195:36361)
I20260807 10:50:34.356086 3367 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 10:50:34.356703 12476 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.73.195@KRBTEST.COM'} at 127.3.73.195:32969
WARNING: no policy specified for oryx/127.3.73.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.73.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.73.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.253.keytab.
Entry for principal oryx/127.3.73.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.253.keytab.
I20260807 10:50:34.429823 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.253:35079
--webserver_interface=127.3.73.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.253.keytab
--principal=oryx/127.3.73.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:35907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:41969,127.3.73.253:35079
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 10:50:34.758044 12938 heartbeater.cc:503] Master 127.3.73.254:41969 was elected leader, sending a full tablet report...
W20260807 10:50:34.829984 13136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:34.830363 13136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:34.830541 13136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:34.838271 13136 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:50:34.838426 13136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:34.838541 13136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:34.838639 13136 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:50:34.838747 13136 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:50:34.848927 13136 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:41969,127.3.73.253:35079
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.253.keytab
--principal=oryx/127.3.73.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.253:35079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.73.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
W20260807 10:50:34.849368 13136 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.3.73.254:41969,127.3.73.253:35079'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 10:50:34.850582 13136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:34.852067 13136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:34.865362 13144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:34.865541 13150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:34.866443 13143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:34.868924 13136 server_base.cc:1034] running on GCE node
I20260807 10:50:34.870318 13136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:34.872212 13136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:34.873544 13136 hybrid_clock.cc:648] HybridClock initialized: now 1786099834873473 us; error 42 us; skew 500 ppm
I20260807 10:50:34.873994 13136 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:34 dist-test-slave-x5fp krb5kdc[12409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099834, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:34.878804 13136 init.cc:377] Logged in from keytab as oryx/127.3.73.253@KRBTEST.COM (short username oryx)
I20260807 10:50:34.881435 13136 webserver.cc:460] Webserver started at http://127.3.73.253:38899/ using document root <none> and password file <none>
I20260807 10:50:34.882280 13136 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:34.882485 13136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:34.883014 13136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:34.887254 13136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-1/data/instance:
uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f"
format_stamp: "Formatted at 2026-08-07 10:50:34 on dist-test-slave-x5fp"
server_key: "7624a8f3e3a731c643e23f4fc957a2ec"
server_key_iv: "d1a085690548be97a8cc49a985aa0b39"
server_key_version: "encryptionkey@0"
I20260807 10:50:34.888244 13136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-1/wal/instance:
uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f"
format_stamp: "Formatted at 2026-08-07 10:50:34 on dist-test-slave-x5fp"
server_key: "7624a8f3e3a731c643e23f4fc957a2ec"
server_key_iv: "d1a085690548be97a8cc49a985aa0b39"
server_key_version: "encryptionkey@0"
I20260807 10:50:34.897037 13136 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.011s sys 0.000s
I20260807 10:50:34.902529 13162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:34.904400 13136 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260807 10:50:34.904685 13136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-1/wal
uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f"
format_stamp: "Formatted at 2026-08-07 10:50:34 on dist-test-slave-x5fp"
server_key: "7624a8f3e3a731c643e23f4fc957a2ec"
server_key_iv: "d1a085690548be97a8cc49a985aa0b39"
server_key_version: "encryptionkey@0"
I20260807 10:50:34.905128 13136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:34.925567 13136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:34.931486 13136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:34.931825 13136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:34.961117 13136 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.253:35079
I20260807 10:50:34.961163 13242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.253:35079 every 8 connection(s)
I20260807 10:50:34.962787 13136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-1/data/info.pb
I20260807 10:50:34.967180 13243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:34.973026 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 13136
I20260807 10:50:34.976267 13243 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 10:50:34 dist-test-slave-x5fp krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.73.253@KRBTEST.COM
Aug 07 10:50:34 dist-test-slave-x5fp krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099834, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.253@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM
I20260807 10:50:34.999282 13250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:34.981259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56590 (local address 127.3.73.254:41969)
0807 10:50:34.982551 (+ 1292us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:34.982556 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:34.982573 (+ 17us) server_negotiation.cc:407] Connection header received
0807 10:50:34.982635 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:34.982638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:34.982699 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:34.982831 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:34.984185 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:34.985040 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:34.987087 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:34.987508 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:34.987651 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:34.992133 (+ 4482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:34.992139 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:34.992142 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:34.992198 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:34.995538 (+ 3340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:34.996502 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:34.996507 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:34.996509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:34.996755 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:34.997261 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:34.997266 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:34.997268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:34.997653 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:34.997774 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:34.998684 (+ 910us) server_negotiation.cc:299] Negotiation successful
0807 10:50:34.998969 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1187,"thread_start_us":1087,"threads_started":1}
I20260807 10:50:35.000988 13249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:34.981153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.0.0.1:56590)
0807 10:50:34.981927 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:34.981980 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:34.982123 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:34.982900 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:34.982916 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:34.983707 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:34.983997 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:34.984006 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:34.985187 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:34.985199 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:34.986936 (+ 1737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:34.986952 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:34.987750 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:34.987757 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:34.987928 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:34.988692 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:34.988821 (+ 129us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:34.991858 (+ 3037us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:34.995732 (+ 3874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:34.995743 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:34.995754 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:34.996337 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:34.996894 (+ 557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:34.996898 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:34.996900 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:34.997115 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:34.997830 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:34.997840 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:34.998083 (+ 243us) client_negotiation.cc:769] Sending connection context
0807 10:50:34.998566 (+ 483us) client_negotiation.cc:240] Negotiation successful
0807 10:50:34.999299 (+ 733us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"mutex_wait_us":47,"thread_start_us":379,"threads_started":1}
I20260807 10:50:35.001025 13248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:34.975738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43280 (local address 127.3.73.253:35079)
0807 10:50:34.976930 (+ 1192us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:34.976942 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:34.976990 (+ 48us) server_negotiation.cc:407] Connection header received
0807 10:50:34.978766 (+ 1776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:34.978790 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:34.979266 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:34.979610 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:34.982054 (+ 2444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:34.983713 (+ 1659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:34.985360 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:34.986454 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:34.986654 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:34.991950 (+ 5296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:34.991966 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:34.991987 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:34.992031 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:34.995323 (+ 3292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:34.996329 (+ 1006us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:34.996339 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:34.996343 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:34.996568 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:34.997154 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:34.997162 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:34.997167 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:34.997922 (+ 755us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:34.998333 (+ 411us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:34.998567 (+ 234us) server_negotiation.cc:299] Negotiation successful
0807 10:50:34.999404 (+ 837us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":187,"server-negotiator.queue_time_us":906,"thread_start_us":523,"threads_started":1}
I20260807 10:50:35.004449 13243 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 10:50:35 dist-test-slave-x5fp krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099834, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.253@KRBTEST.COM for oryx/127.3.73.253@KRBTEST.COM
I20260807 10:50:35.019565 13249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:35.005351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:35079 (local address 127.0.0.1:43282)
0807 10:50:35.005604 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:35.005631 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:35.005790 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:35.006212 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:35.006217 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:35.006628 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:35.006776 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:35.006785 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.007880 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.007887 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:35.008833 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:35.008846 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.009581 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.009586 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:35.010171 (+ 585us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:35.010747 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:35.010764 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:35.013131 (+ 2367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:35.016473 (+ 3342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:35.016481 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:35.016489 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:35.017140 (+ 651us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:35.017597 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:35.017602 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:35.017608 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:35.017745 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:35.018370 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:35.018376 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:35.018538 (+ 162us) client_negotiation.cc:769] Sending connection context
0807 10:50:35.018690 (+ 152us) client_negotiation.cc:240] Negotiation successful
0807 10:50:35.019062 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260807 10:50:35.019595 13248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:35.005615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43282 (local address 127.3.73.253:35079)
0807 10:50:35.005786 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:35.005792 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:35.005811 (+ 19us) server_negotiation.cc:407] Connection header received
0807 10:50:35.005900 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:35.005907 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:35.005981 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:35.006188 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:35.006932 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.007689 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.008969 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.009425 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.009571 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:35.013344 (+ 3773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:35.013349 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:35.013352 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:35.013382 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:35.016280 (+ 2898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:35.017276 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:35.017284 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:35.017288 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:35.017457 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:35.017875 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:35.017885 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:35.017887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:35.018199 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:35.018397 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:35.018789 (+ 392us) server_negotiation.cc:299] Negotiation successful
0807 10:50:35.019215 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":187,"server-negotiator.queue_time_us":66}
I20260807 10:50:35.025385 13243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f: Bootstrap starting.
I20260807 10:50:35.029004 13243 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:35.030313 13243 log.cc:824] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f: Log is configured to *not* fsync() on all Append() calls
I20260807 10:50:35.033689 13243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f: No bootstrap required, opened a new log
I20260807 10:50:35.041956 13243 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } }
I20260807 10:50:35.042389 13243 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:35.042518 13243 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33fdc900cfa34b11aa90dc7c2cf3c84f, State: Initialized, Role: FOLLOWER
I20260807 10:50:35.043259 13243 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } }
I20260807 10:50:35.044358 13252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } } }
I20260807 10:50:35.044782 13252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 10:50:35.045651 13243 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [sys.catalog]: configured and running, proceeding with master startup.
W20260807 10:50:35.057252 13269 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 10:50:35.057476 13269 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 10:50:35.061929 13136 master_runner.cc:428] Detected that this master 33fdc900cfa34b11aa90dc7c2cf3c84f is joining an existing cluster
I20260807 10:50:35.062050 13136 master_runner.cc:432] Initiating AddMaster RPC to add 127.3.73.253:35079
I20260807 10:50:35.090926 13250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:35.071770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56604 (local address 127.3.73.254:41969)
0807 10:50:35.072003 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:35.072012 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:35.072833 (+ 821us) server_negotiation.cc:407] Connection header received
0807 10:50:35.073135 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:35.073142 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:35.073251 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:35.073331 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:35.075206 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.076355 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.078675 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.079342 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.079449 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:35.082371 (+ 2922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:35.082380 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:35.082388 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:35.082436 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:35.086590 (+ 4154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:35.087521 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:35.087527 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:35.087531 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:35.087708 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:35.088256 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:35.088262 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:35.088267 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:35.088673 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:35.089172 (+ 499us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:35.089917 (+ 745us) server_negotiation.cc:299] Negotiation successful
0807 10:50:35.090205 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 10:50:35.092937 13277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:35.071786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.0.0.1:56604)
0807 10:50:35.072673 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:35.072694 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:35.072962 (+ 268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:35.073582 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:35.073586 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:35.074513 (+ 927us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:35.075001 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:35.075026 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.076604 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.076610 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:35.078483 (+ 1873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:35.078499 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.079781 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.079789 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:35.079992 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:35.080792 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:35.080821 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:35.082128 (+ 1307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:35.086780 (+ 4652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:35.086791 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:35.086795 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:35.087346 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:35.087898 (+ 552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:35.087903 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:35.087906 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:35.088106 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:35.089266 (+ 1160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:35.089271 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:35.089507 (+ 236us) client_negotiation.cc:769] Sending connection context
0807 10:50:35.091571 (+ 2064us) client_negotiation.cc:240] Negotiation successful
0807 10:50:35.092309 (+ 738us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"mutex_wait_us":77,"thread_start_us":508,"threads_started":1}
I20260807 10:50:35.097635 13248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:35.073320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43288 (local address 127.3.73.253:35079)
0807 10:50:35.073508 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:35.073516 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:35.074753 (+ 1237us) server_negotiation.cc:407] Connection header received
0807 10:50:35.075020 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:35.075027 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:35.075100 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:35.075165 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:35.076622 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.078015 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.080199 (+ 2184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.081079 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.081188 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:35.088066 (+ 6878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:35.088074 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:35.088082 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:35.088124 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:35.092524 (+ 4400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:35.094778 (+ 2254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:35.094797 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:35.094801 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:35.095027 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:35.095606 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:35.095613 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:35.095618 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:35.096089 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:35.096222 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:35.096847 (+ 625us) server_negotiation.cc:299] Negotiation successful
0807 10:50:35.097134 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":65}
WARNING: no policy specified for oryx/127.3.73.252@KRBTEST.COM; defaulting to no policy
I20260807 10:50:35.100195 13278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:35.073639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:35079 (local address 127.0.0.1:43288)
0807 10:50:35.074636 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:35.074663 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:35.074867 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:35.075387 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:35.075394 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:35.076170 (+ 776us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:35.076471 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:35.076481 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.078233 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.078241 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:35.080019 (+ 1778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:35.080039 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.081544 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.081553 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:35.081787 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:35.083370 (+ 1583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:35.083404 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:35.087836 (+ 4432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:35.092701 (+ 4865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:35.092706 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:35.092710 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:35.093344 (+ 634us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:35.095215 (+ 1871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:35.095222 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:35.095227 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:35.095445 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:35.096304 (+ 859us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:35.096308 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:35.096472 (+ 164us) client_negotiation.cc:769] Sending connection context
0807 10:50:35.098207 (+ 1735us) client_negotiation.cc:240] Negotiation successful
0807 10:50:35.098568 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":837,"thread_start_us":602,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.73.252@KRBTEST.COM" created.
Aug 07 10:50:35 dist-test-slave-x5fp krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099832, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.254@KRBTEST.COM for oryx/127.3.73.253@KRBTEST.COM
I20260807 10:50:35.121996 13248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:35.100981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43290 (local address 127.3.73.253:35079)
0807 10:50:35.101178 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:35.101187 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:35.101897 (+ 710us) server_negotiation.cc:407] Connection header received
0807 10:50:35.102898 (+ 1001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:35.102902 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:35.102994 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:35.103107 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:35.104280 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.105085 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.106427 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.106870 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.107006 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:35.113750 (+ 6744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:35.113760 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:35.113764 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:35.113804 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:35.117833 (+ 4029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:35.119039 (+ 1206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:35.119049 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:35.119055 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:35.119280 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:35.119880 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:35.119887 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:35.119891 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:35.120305 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:35.120508 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:35.121215 (+ 707us) server_negotiation.cc:299] Negotiation successful
0807 10:50:35.121537 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 10:50:35.122205 13283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:35.100978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:35079 (local address 127.0.0.1:43290)
0807 10:50:35.101761 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:35.101809 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:35.102609 (+ 800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:35.103167 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:35.103175 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:35.103888 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:35.104116 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:35.104128 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.105233 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.105245 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:35.106256 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:35.106269 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.107055 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.107060 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:35.107276 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:35.110770 (+ 3494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:35.110793 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:35.113456 (+ 2663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:35.118030 (+ 4574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:35.118054 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:35.118066 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:35.118813 (+ 747us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:35.119459 (+ 646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:35.119468 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:35.119475 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:35.119718 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:35.120485 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:35.120499 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:35.120804 (+ 305us) client_negotiation.cc:769] Sending connection context
0807 10:50:35.121071 (+ 267us) client_negotiation.cc:240] Negotiation successful
0807 10:50:35.121597 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":377,"threads_started":1}
I20260807 10:50:35.125407 12476 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.3.73.253:35079: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f"
member_type: NON_VOTER
last_known_addr {
host: "127.3.73.253"
port: 35079
}
attrs {
promote: true
}
}
dest_uuid: "ed8559261108476a81169c92d7e720db"
cas_config_opid_index: -1
I20260807 10:50:35.126892 12476 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: NON_VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: true } }
I20260807 10:50:35.139744 13289 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 33fdc900cfa34b11aa90dc7c2cf3c84f (127.3.73.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: NON_VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: true } } }
I20260807 10:50:35.140688 13205 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:35.145542 13205 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [term 1 FOLLOWER]: Refusing update from remote peer ed8559261108476a81169c92d7e720db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260807 10:50:35.145793 13289 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.3.73.253:35079
I20260807 10:50:35.146938 13136 master.cc:537] Master@127.3.73.253:35079 shutting down...
I20260807 10:50:35.146898 13287 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ed8559261108476a81169c92d7e720db" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: NON_VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: true } } }
I20260807 10:50:35.148852 13287 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:35.149600 13289 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [LEADER]: Connected to new peer: Peer: permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: NON_VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.003s
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.73.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.252.keytab.
Entry for principal oryx/127.3.73.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.252.keytab.
I20260807 10:50:35.153054 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.252:42805
--webserver_interface=127.3.73.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.252.keytab
--principal=oryx/127.3.73.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:35907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:41969,127.3.73.253:35079,127.3.73.252:42805
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:50:35.155622 12457 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260807 10:50:35.170034 12457 consensus_peers.cc:487] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db -> Peer 33fdc900cfa34b11aa90dc7c2cf3c84f (127.3.73.253:35079): Couldn't send request to peer 33fdc900cfa34b11aa90dc7c2cf3c84f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 10:50:35.170253 13136 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:50:35.170953 13136 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:50:35.171191 13136 tablet_replica.cc:330] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f: stopping tablet replica
I20260807 10:50:35.179119 13136 master.cc:552] Master@127.3.73.253:35079 shutdown complete.
I20260807 10:50:35.179266 13136 master_runner.cc:305] Clearing existing system tablet
I20260807 10:50:35.179859 13136 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:35.180048 13136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:35.183888 13136 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 10:50:35.186022 13303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:35.186503 13136 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:35.186812 13136 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 10:50:35.186936 13136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-1/wal
uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f"
format_stamp: "Formatted at 2026-08-07 10:50:34 on dist-test-slave-x5fp"
server_key: "7624a8f3e3a731c643e23f4fc957a2ec"
server_key_iv: "d1a085690548be97a8cc49a985aa0b39"
server_key_version: "encryptionkey@0"
I20260807 10:50:35.193243 13136 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:50:35.201109 13136 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 10:50:35.201335 13136 log.cc:1196] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 10:50:35.201846 13136 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f: Deleting consensus metadata
I20260807 10:50:35.202148 13136 master_runner.cc:315] Copying system tablet from 127.3.73.254:41969
I20260807 10:50:35.205456 13136 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f: tablet copy: Beginning tablet copy session from remote peer at address 127.3.73.254:41969
I20260807 10:50:35.226948 13250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:35.206692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56620 (local address 127.3.73.254:41969)
0807 10:50:35.206896 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:35.206902 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:35.209334 (+ 2432us) server_negotiation.cc:407] Connection header received
0807 10:50:35.209574 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:35.209580 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:35.209679 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:35.209808 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:35.210981 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.212255 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.214223 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.214956 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.215062 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:35.217987 (+ 2925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:35.217994 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:35.217999 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:35.218046 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:35.222427 (+ 4381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:35.223490 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:35.223497 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:35.223501 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:35.223688 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:35.224323 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:35.224328 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:35.224332 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:35.225091 (+ 759us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:35.225292 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:35.226094 (+ 802us) server_negotiation.cc:299] Negotiation successful
0807 10:50:35.226438 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 10:50:35.228482 13311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:35.206518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.0.0.1:56620)
0807 10:50:35.209175 (+ 2657us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:35.209198 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:35.209399 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:35.209856 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:35.209861 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:35.210488 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:35.210785 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:35.210800 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.212453 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.212459 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:35.214023 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:35.214040 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.215364 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.215380 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:35.215601 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:35.216415 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:35.216443 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:35.217718 (+ 1275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:35.222639 (+ 4921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:35.222647 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:35.222651 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:35.223291 (+ 640us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:35.223875 (+ 584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:35.223881 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:35.223888 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:35.224135 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:35.225450 (+ 1315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:35.225457 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:35.225707 (+ 250us) client_negotiation.cc:769] Sending connection context
0807 10:50:35.227537 (+ 1830us) client_negotiation.cc:240] Negotiation successful
0807 10:50:35.227931 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2488,"thread_start_us":2366,"threads_started":1}
I20260807 10:50:35.229969 12513 tablet_copy_service.cc:140] P ed8559261108476a81169c92d7e720db: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 33fdc900cfa34b11aa90dc7c2cf3c84f ({username='oryx', principal='oryx/127.3.73.253@KRBTEST.COM'} at 127.0.0.1:56620)
I20260807 10:50:35.230445 12513 tablet_copy_service.cc:161] P ed8559261108476a81169c92d7e720db: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 33fdc900cfa34b11aa90dc7c2cf3c84f at {username='oryx', principal='oryx/127.3.73.253@KRBTEST.COM'} at 127.0.0.1:56620: session id = 33fdc900cfa34b11aa90dc7c2cf3c84f-00000000000000000000000000000000
I20260807 10:50:35.237660 12513 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 10:50:35.241259 13136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 10:50:35.249193 13136 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f: tablet copy: Starting download of 0 data blocks...
I20260807 10:50:35.249691 13136 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f: tablet copy: Starting download of 1 WAL segments...
I20260807 10:50:35.253986 13136 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 10:50:35.259351 12513 tablet_copy_service.cc:342] P ed8559261108476a81169c92d7e720db: Request end of tablet copy session 33fdc900cfa34b11aa90dc7c2cf3c84f-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.3.73.253@KRBTEST.COM'} at 127.0.0.1:56620
I20260807 10:50:35.259683 12513 tablet_copy_service.cc:433] P ed8559261108476a81169c92d7e720db: ending tablet copy session 33fdc900cfa34b11aa90dc7c2cf3c84f-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 33fdc900cfa34b11aa90dc7c2cf3c84f
I20260807 10:50:35.274731 13136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:35.278260 13312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:35.278800 13313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:35.280638 13316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:35.282572 13136 server_base.cc:1034] running on GCE node
I20260807 10:50:35.282960 13136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:35.283656 13136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:35.284837 13136 hybrid_clock.cc:648] HybridClock initialized: now 1786099835284769 us; error 44 us; skew 500 ppm
I20260807 10:50:35.285091 13136 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:35 dist-test-slave-x5fp krb5kdc[12409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099835, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:35.287403 13136 init.cc:377] Logged in from keytab as oryx/127.3.73.253@KRBTEST.COM (short username oryx)
I20260807 10:50:35.293944 13136 webserver.cc:460] Webserver started at http://127.3.73.253:44513/ using document root <none> and password file <none>
I20260807 10:50:35.294425 13136 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:35.294534 13136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:35.298509 13136 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 10:50:35.307940 13326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:35.309274 13136 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.002s sys 0.005s
I20260807 10:50:35.309496 13136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-1/wal
uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f"
format_stamp: "Formatted at 2026-08-07 10:50:34 on dist-test-slave-x5fp"
server_key: "7624a8f3e3a731c643e23f4fc957a2ec"
server_key_iv: "d1a085690548be97a8cc49a985aa0b39"
server_key_version: "encryptionkey@0"
I20260807 10:50:35.309752 13136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:35.360108 13124 heartbeater.cc:503] Master 127.3.73.254:41969 was elected leader, sending a full tablet report...
I20260807 10:50:35.376312 13136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:35.380926 13136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:35.475950 13136 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.253:35079
I20260807 10:50:35.476040 13407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.253:35079 every 8 connection(s)
I20260807 10:50:35.477276 13136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-1/data/info.pb
I20260807 10:50:35.484515 13408 sys_catalog.cc:263] Verifying existing consensus state
I20260807 10:50:35.489454 13408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f: Bootstrap starting.
I20260807 10:50:35.511140 13408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 10:50:35.511843 13408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f: Bootstrap complete.
I20260807 10:50:35.513130 13408 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: NON_VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: true } }
I20260807 10:50:35.513417 13408 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 33fdc900cfa34b11aa90dc7c2cf3c84f, State: Initialized, Role: LEARNER
I20260807 10:50:35.513743 13408 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: NON_VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: true } }
I20260807 10:50:35.514962 13408 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:50:35.514735 13413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: NON_VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: true } } }
I20260807 10:50:35.515202 13413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [sys.catalog]: This master's current role is: LEARNER
I20260807 10:50:35.527819 13426 catalog_manager.cc:1259] Loaded cluster ID: 91f30ac1709340818f64a197b4f10202
I20260807 10:50:35.527933 13426 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f: loading cluster ID for follower catalog manager: success
I20260807 10:50:35.531778 13426 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f: acquiring CA information for follower catalog manager: success
I20260807 10:50:35.534464 13426 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 10:50:35.715911 13299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:35.716390 13299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:35.716655 13299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:35.728485 13299 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:50:35.728634 13299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:35.728700 13299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:35.728921 13299 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:50:35.729094 13299 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:50:35.744076 13299 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:41969,127.3.73.253:35079,127.3.73.252:42805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.252.keytab
--principal=oryx/127.3.73.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.252:42805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.73.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:35.745930 13299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:35.747682 13299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 10:50:35.770138 13437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:35.757860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:35079 (local address 127.0.0.1:43298)
0807 10:50:35.758546 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:35.758586 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:35.758848 (+ 262us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:35.760365 (+ 1517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:35.760371 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:35.760406 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:50:35.760637 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:35.760651 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.763208 (+ 2557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.763213 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:35.765382 (+ 2169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:35.765420 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.768679 (+ 3259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.768690 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:35.769096 (+ 406us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:35.769263 (+ 167us) client_negotiation.cc:769] Sending connection context
0807 10:50:35.769386 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 10:50:35.769599 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":409,"threads_started":1}
W20260807 10:50:35.769958 13434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:35.770222 13431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:35.770591 13436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:35.772138 13438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:35.758416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43298 (local address 127.3.73.253:35079)
0807 10:50:35.759807 (+ 1391us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:35.759814 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:35.759857 (+ 43us) server_negotiation.cc:407] Connection header received
0807 10:50:35.759942 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:35.759947 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:35.760065 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:35.760881 (+ 816us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:50:35.761178 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.762997 (+ 1819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.765646 (+ 2649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.768311 (+ 2665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.770728 (+ 2417us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:35.771176 (+ 448us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:35.771293 (+ 117us) server_negotiation.cc:299] Negotiation successful
0807 10:50:35.771480 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1167,"thread_start_us":950,"threads_started":1}
I20260807 10:50:35.772377 13299 server_base.cc:1034] running on GCE node
I20260807 10:50:35.773763 13299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:35.773741 13368 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260807 10:50:35.774470 13413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed8559261108476a81169c92d7e720db. Latest consensus state: current_term: 1 leader_uuid: "ed8559261108476a81169c92d7e720db" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: NON_VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: true } } }
I20260807 10:50:35.774963 13413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [sys.catalog]: This master's current role is: LEARNER
I20260807 10:50:35.775985 13299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:35.777281 13299 hybrid_clock.cc:648] HybridClock initialized: now 1786099835777136 us; error 58 us; skew 500 ppm
I20260807 10:50:35.777765 13299 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:35 dist-test-slave-x5fp krb5kdc[12409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099835, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:35.782788 13299 init.cc:377] Logged in from keytab as oryx/127.3.73.252@KRBTEST.COM (short username oryx)
I20260807 10:50:35.785796 13299 webserver.cc:460] Webserver started at http://127.3.73.252:40565/ using document root <none> and password file <none>
I20260807 10:50:35.786823 13299 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:35.787041 13299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:35.787484 13299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:35.791905 13299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-2/data/instance:
uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e"
format_stamp: "Formatted at 2026-08-07 10:50:35 on dist-test-slave-x5fp"
server_key: "6442a5fbaec5c25da3c28256bc0c7669"
server_key_iv: "37b0bc6d56f30bdfde91ad537c17709f"
server_key_version: "encryptionkey@0"
I20260807 10:50:35.793114 13299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-2/wal/instance:
uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e"
format_stamp: "Formatted at 2026-08-07 10:50:35 on dist-test-slave-x5fp"
server_key: "6442a5fbaec5c25da3c28256bc0c7669"
server_key_iv: "37b0bc6d56f30bdfde91ad537c17709f"
server_key_version: "encryptionkey@0"
I20260807 10:50:35.800189 13299 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260807 10:50:35.806116 13448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:35.807822 13299 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260807 10:50:35.808054 13299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-2/wal
uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e"
format_stamp: "Formatted at 2026-08-07 10:50:35 on dist-test-slave-x5fp"
server_key: "6442a5fbaec5c25da3c28256bc0c7669"
server_key_iv: "37b0bc6d56f30bdfde91ad537c17709f"
server_key_version: "encryptionkey@0"
I20260807 10:50:35.808291 13299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:35.827598 13299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:35.834573 13299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:35.835080 13299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:35.886191 13299 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.252:42805
I20260807 10:50:35.886364 13525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.252:42805 every 8 connection(s)
I20260807 10:50:35.888315 13299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-2/data/info.pb
I20260807 10:50:35.891521 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 13299
I20260807 10:50:35.896436 13526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:35.911594 13526 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 10:50:35 dist-test-slave-x5fp krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.73.252@KRBTEST.COM
I20260807 10:50:35.934360 13534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:35.894537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45698 (local address 127.3.73.252:42805)
0807 10:50:35.896905 (+ 2368us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:35.896921 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:35.897015 (+ 94us) server_negotiation.cc:407] Connection header received
0807 10:50:35.900050 (+ 3035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:35.900081 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:35.900597 (+ 516us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:35.901138 (+ 541us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:35.905177 (+ 4039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.907424 (+ 2247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.910347 (+ 2923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.911575 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.911776 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:35.920448 (+ 8672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:35.920463 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:35.920489 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:35.920553 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:35.927755 (+ 7202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:35.929168 (+ 1413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:35.929176 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:35.929184 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:35.929534 (+ 350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:35.930075 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:35.930082 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:35.930099 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:35.931097 (+ 998us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:35.931437 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:35.932185 (+ 748us) server_negotiation.cc:299] Negotiation successful
0807 10:50:35.932714 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1800,"thread_start_us":1178,"threads_started":1}
Aug 07 10:50:35 dist-test-slave-x5fp krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099835, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.252@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM
I20260807 10:50:35.945631 13536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:35.916951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.0.0.1:56628)
0807 10:50:35.919684 (+ 2733us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:35.919738 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:35.920047 (+ 309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:35.920667 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:35.920673 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:35.922463 (+ 1790us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:35.923485 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:35.923509 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.926620 (+ 3111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.926625 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:35.929265 (+ 2640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:35.929280 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.930904 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.930912 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:35.931116 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:35.932220 (+ 1104us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:35.932244 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:35.936404 (+ 4160us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:35.941657 (+ 5253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:35.941678 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:35.941683 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:35.942354 (+ 671us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:35.942934 (+ 580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:35.942938 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:35.942941 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:35.943296 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:35.944144 (+ 848us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:35.944151 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:35.944388 (+ 237us) client_negotiation.cc:769] Sending connection context
0807 10:50:35.944664 (+ 276us) client_negotiation.cc:240] Negotiation successful
0807 10:50:35.945096 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2270,"thread_start_us":1863,"threads_started":1}
I20260807 10:50:35.945673 13535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:35.916853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56628 (local address 127.3.73.254:41969)
0807 10:50:35.918030 (+ 1177us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:35.918042 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:35.919896 (+ 1854us) server_negotiation.cc:407] Connection header received
0807 10:50:35.920235 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:35.920243 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:35.920437 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:35.920577 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:35.923959 (+ 3382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.926335 (+ 2376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.929434 (+ 3099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.930545 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.930706 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:35.936832 (+ 6126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:35.936841 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:35.936845 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:35.936887 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:35.941432 (+ 4545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:35.942529 (+ 1097us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:35.942534 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:35.942537 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:35.942731 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:35.943441 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:35.943445 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:35.943447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:35.943985 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:35.944114 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:35.944974 (+ 860us) server_negotiation.cc:299] Negotiation successful
0807 10:50:35.945171 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":871,"thread_start_us":636,"threads_started":1}
I20260807 10:50:35.947597 13526 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 10:50:35 dist-test-slave-x5fp krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099835, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.252@KRBTEST.COM for oryx/127.3.73.253@KRBTEST.COM
I20260807 10:50:35.969144 13536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:35.948401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:35079 (local address 127.0.0.1:43312)
0807 10:50:35.948649 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:35.948672 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:35.948914 (+ 242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:35.949447 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:35.949452 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:35.950057 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:35.950419 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:35.950430 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.951928 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.951938 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:35.953369 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:35.953384 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.954477 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.954485 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:35.955107 (+ 622us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:35.956477 (+ 1370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:35.956503 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:35.959791 (+ 3288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:35.965679 (+ 5888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:35.965684 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:35.965689 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:35.966568 (+ 879us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:35.967238 (+ 670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:35.967243 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:35.967247 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:35.967487 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:35.968141 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:35.968144 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:35.968319 (+ 175us) client_negotiation.cc:769] Sending connection context
0807 10:50:35.968433 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 10:50:35.968821 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 10:50:35.969491 13438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:35.948879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43312 (local address 127.3.73.253:35079)
0807 10:50:35.949113 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:35.949118 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:35.949135 (+ 17us) server_negotiation.cc:407] Connection header received
0807 10:50:35.949189 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:35.949224 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:35.949294 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:35.949417 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:35.950604 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.951721 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.953538 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.954208 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.954422 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:35.960097 (+ 5675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:35.960103 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:35.960106 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:35.960146 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:35.965502 (+ 5356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:35.966784 (+ 1282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:35.966792 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:35.966798 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:35.967072 (+ 274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:35.967632 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:35.967636 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:35.967639 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:35.968028 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:35.968205 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:35.968555 (+ 350us) server_negotiation.cc:299] Negotiation successful
0807 10:50:35.968913 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 10:50:35.970705 13526 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42805 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 10:50:35 dist-test-slave-x5fp krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099835, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.252@KRBTEST.COM for oryx/127.3.73.252@KRBTEST.COM
I20260807 10:50:35.988515 13536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:35.971509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:42805 (local address 127.0.0.1:45708)
0807 10:50:35.971857 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:35.971880 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:35.972020 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:35.972385 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:35.972388 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:35.973040 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:35.973260 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:35.973270 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.974526 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.974530 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:35.975732 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:35.975745 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.976862 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.976868 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:35.977110 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:35.977957 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:35.977976 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:35.980938 (+ 2962us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:35.985107 (+ 4169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:35.985111 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:35.985114 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:35.985724 (+ 610us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:35.986249 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:35.986252 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:35.986255 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:35.986543 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:35.987291 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:35.987295 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:35.987537 (+ 242us) client_negotiation.cc:769] Sending connection context
0807 10:50:35.987694 (+ 157us) client_negotiation.cc:240] Negotiation successful
0807 10:50:35.987914 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 10:50:35.988757 13534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:35.971953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45708 (local address 127.3.73.252:42805)
0807 10:50:35.972128 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:35.972132 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:35.972149 (+ 17us) server_negotiation.cc:407] Connection header received
0807 10:50:35.972192 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:35.972195 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:35.972256 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:35.972343 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:35.973397 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.974381 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.975882 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:35.976578 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:35.976685 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:35.981145 (+ 4460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:35.981150 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:35.981154 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:35.981184 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:35.984917 (+ 3733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:35.985892 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:35.985896 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:35.985898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:35.986138 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:35.986698 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:35.986702 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:35.986704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:35.987164 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:35.987299 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:35.987911 (+ 612us) server_negotiation.cc:299] Negotiation successful
0807 10:50:35.988176 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 10:50:35.995635 13526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e: Bootstrap starting.
I20260807 10:50:36.000692 13526 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:36.003015 13526 log.cc:824] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e: Log is configured to *not* fsync() on all Append() calls
I20260807 10:50:36.007483 13526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e: No bootstrap required, opened a new log
I20260807 10:50:36.019665 13526 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } } peers { permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42805 } }
I20260807 10:50:36.020413 13526 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:36.020609 13526 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c3e3fe8454b40ecafbcc66cee29bd5e, State: Initialized, Role: FOLLOWER
I20260807 10:50:36.021425 13526 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } } peers { permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42805 } }
I20260807 10:50:36.023030 13539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } } peers { permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42805 } } }
I20260807 10:50:36.023747 13539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 10:50:36.024137 13526 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:50:36.043422 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 12571
W20260807 10:50:36.044109 13557 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 10:50:36.044368 13557 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 10:50:36.048318 13299 master_runner.cc:428] Detected that this master 0c3e3fe8454b40ecafbcc66cee29bd5e is joining an existing cluster
I20260807 10:50:36.048558 13299 master_runner.cc:432] Initiating AddMaster RPC to add 127.3.73.252:42805
Aug 07 10:50:36 dist-test-slave-x5fp krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.73.193@KRBTEST.COM
I20260807 10:50:36.083531 13559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:36.045592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44336 (local address 127.3.73.193:33371)
0807 10:50:36.047035 (+ 1443us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:36.047043 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:36.047070 (+ 27us) server_negotiation.cc:407] Connection header received
0807 10:50:36.047130 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:36.047136 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:36.047341 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:36.047537 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:36.050132 (+ 2595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:36.052700 (+ 2568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:36.056029 (+ 3329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:36.057253 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:36.058822 (+ 1569us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:36.068199 (+ 9377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:36.068227 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:36.068257 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:36.068333 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:36.075309 (+ 6976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:36.077064 (+ 1755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:36.077080 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:36.077087 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:36.077519 (+ 432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:36.078439 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:36.078445 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:36.078449 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:36.079238 (+ 789us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:36.079763 (+ 525us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:36.081677 (+ 1914us) server_negotiation.cc:299] Negotiation successful
0807 10:50:36.082567 (+ 890us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1250,"thread_start_us":862,"threads_started":1}
I20260807 10:50:36.085537 12740 generic_service.cc:196] Checking for leaks (request via RPC)
W20260807 10:50:36.092113 13457 tablet.cc:2369] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:50:36.101488 13535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:36.058527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56630 (local address 127.3.73.254:41969)
0807 10:50:36.058737 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:36.058743 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:36.070299 (+ 11556us) server_negotiation.cc:407] Connection header received
0807 10:50:36.070851 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:36.070857 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:36.070977 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:36.071138 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:36.074009 (+ 2871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:36.076010 (+ 2001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:36.079559 (+ 3549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:36.080548 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:36.080705 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:36.087780 (+ 7075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:36.087789 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:36.087795 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:36.087849 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:36.095715 (+ 7866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:36.097391 (+ 1676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:36.097398 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:36.097402 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:36.097773 (+ 371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:36.098591 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:36.098598 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:36.098602 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:36.099175 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:36.099523 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:36.100609 (+ 1086us) server_negotiation.cc:299] Negotiation successful
0807 10:50:36.100949 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 10:50:36.103160 13567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:36.061913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.0.0.1:56630)
0807 10:50:36.069761 (+ 7848us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:36.069787 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:36.070643 (+ 856us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:36.071166 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:36.071173 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:36.072782 (+ 1609us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:36.073719 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:36.073739 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:36.076288 (+ 2549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:36.076294 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:36.079317 (+ 3023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:36.079332 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:36.081284 (+ 1952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:36.081306 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:36.081519 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:36.083015 (+ 1496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:36.083044 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:36.085249 (+ 2205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:36.096079 (+ 10830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:36.096097 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:36.096102 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:36.097174 (+ 1072us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:36.098019 (+ 845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:36.098025 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:36.098029 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:36.098382 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:36.099601 (+ 1219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:36.099606 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:36.099931 (+ 325us) client_negotiation.cc:769] Sending connection context
0807 10:50:36.100108 (+ 177us) client_negotiation.cc:240] Negotiation successful
0807 10:50:36.102159 (+ 2051us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7639,"thread_start_us":7246,"threads_started":1}
I20260807 10:50:36.112313 13534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:36.058035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45712 (local address 127.3.73.252:42805)
0807 10:50:36.058333 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:36.058340 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:36.061969 (+ 3629us) server_negotiation.cc:407] Connection header received
0807 10:50:36.062303 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:36.062308 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:36.062401 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:36.062498 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:36.065211 (+ 2713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:36.066704 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:36.079114 (+ 12410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:36.082370 (+ 3256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:36.082529 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:36.100004 (+ 17475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:36.100012 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:36.100018 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:36.100056 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:36.107076 (+ 7020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:36.108342 (+ 1266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:36.108350 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:36.108355 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:36.108672 (+ 317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:36.109363 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:36.109369 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:36.109373 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:36.110099 (+ 726us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:36.110289 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:36.111269 (+ 980us) server_negotiation.cc:299] Negotiation successful
0807 10:50:36.111859 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":295,"server-negotiator.queue_time_us":83}
I20260807 10:50:36.117574 13566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:36.058794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:42805 (local address 127.0.0.1:45712)
0807 10:50:36.061843 (+ 3049us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:36.061881 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:36.062128 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:36.062673 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:36.062678 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:36.064063 (+ 1385us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:36.064996 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:36.065026 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:36.067181 (+ 2155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:36.067187 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:36.077499 (+ 10312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:36.077518 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:36.085702 (+ 8184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:36.085714 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:36.086022 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:36.097682 (+ 11660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:36.097713 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:36.099740 (+ 2027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:36.107282 (+ 7542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:36.107289 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:36.107295 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:36.108178 (+ 883us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:36.108887 (+ 709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:36.108894 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:36.108909 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:36.109217 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:36.110367 (+ 1150us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:36.110372 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:36.110616 (+ 244us) client_negotiation.cc:769] Sending connection context
0807 10:50:36.110751 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 10:50:36.116796 (+ 6045us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2197,"thread_start_us":1591,"threads_started":1}
Aug 07 10:50:36 dist-test-slave-x5fp krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099832, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.254@KRBTEST.COM for oryx/127.3.73.252@KRBTEST.COM
I20260807 10:50:36.140812 13438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:36.070098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43326 (local address 127.3.73.253:35079)
0807 10:50:36.070495 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:36.070502 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:36.087458 (+ 16956us) server_negotiation.cc:407] Connection header received
0807 10:50:36.087607 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:36.087616 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:36.087740 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:36.087874 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:36.098511 (+ 10637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:36.100110 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:36.124207 (+ 24097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:36.125449 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:36.125644 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:36.129765 (+ 4121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:36.129773 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:36.129778 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:36.129827 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:36.135346 (+ 5519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:36.136800 (+ 1454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:36.136806 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:36.136809 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:36.137005 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:36.137604 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:36.137625 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:36.137628 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:36.138197 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:36.138331 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:36.139563 (+ 1232us) server_negotiation.cc:299] Negotiation successful
0807 10:50:36.140048 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244}
I20260807 10:50:36.142118 13534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:36.112812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45720 (local address 127.3.73.252:42805)
0807 10:50:36.113035 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:36.113041 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:36.113068 (+ 27us) server_negotiation.cc:407] Connection header received
0807 10:50:36.113168 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:36.113174 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:36.113284 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:36.114609 (+ 1325us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:36.114912 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:36.116628 (+ 1716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:36.120509 (+ 3881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:36.123448 (+ 2939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:36.123605 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:36.130360 (+ 6755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:36.130368 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:36.130373 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:36.130411 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:36.136655 (+ 6244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:36.137937 (+ 1282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:36.137943 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:36.137947 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:36.138158 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:36.138713 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:36.138718 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:36.138722 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:36.139301 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:36.139434 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:36.141291 (+ 1857us) server_negotiation.cc:299] Negotiation successful
0807 10:50:36.141548 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"spinlock_wait_cycles":30464,"thread_start_us":13122,"threads_started":1}
I20260807 10:50:36.142479 13437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:36.111605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:42805 (local address 127.0.0.1:45720)
0807 10:50:36.112112 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:36.112132 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:36.112269 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:36.113476 (+ 1207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:36.113486 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:36.114086 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:36.114391 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:36.114407 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:36.118079 (+ 3672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:36.118086 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:36.120224 (+ 2138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:36.120246 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:36.124301 (+ 4055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:36.124309 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:36.124630 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:36.125611 (+ 981us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:36.125636 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:36.130129 (+ 4493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:36.136917 (+ 6788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:36.136923 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:36.136927 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:36.137784 (+ 857us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:36.138296 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:36.138301 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:36.138305 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:36.138574 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:36.139508 (+ 934us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:36.139514 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:36.139819 (+ 305us) client_negotiation.cc:769] Sending connection context
0807 10:50:36.139968 (+ 149us) client_negotiation.cc:240] Negotiation successful
0807 10:50:36.140306 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339}
I20260807 10:50:36.142737 13569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:36.071844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:35079 (local address 127.0.0.1:43326)
0807 10:50:36.087242 (+ 15398us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:36.087268 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:36.087463 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:36.088192 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:36.088199 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:36.089951 (+ 1752us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:36.098209 (+ 8258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:36.098226 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:36.101690 (+ 3464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:36.101696 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:36.123448 (+ 21752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:36.123460 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:36.126389 (+ 2929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:36.126424 (+ 35us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:36.126803 (+ 379us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:36.127500 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:36.127526 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:36.129379 (+ 1853us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:36.135619 (+ 6240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:36.135626 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:36.135630 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:36.136577 (+ 947us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:36.137172 (+ 595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:36.137177 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:36.137180 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:36.137449 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:36.138698 (+ 1249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:36.138704 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:36.139119 (+ 415us) client_negotiation.cc:769] Sending connection context
0807 10:50:36.141204 (+ 2085us) client_negotiation.cc:240] Negotiation successful
0807 10:50:36.141889 (+ 685us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15087,"mutex_wait_us":108,"thread_start_us":14633,"threads_started":1}
I20260807 10:50:36.145779 12476 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.3.73.252:42805: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e"
member_type: NON_VOTER
last_known_addr {
host: "127.3.73.252"
port: 42805
}
attrs {
promote: true
}
}
dest_uuid: "ed8559261108476a81169c92d7e720db"
cas_config_opid_index: 5
I20260807 10:50:36.147516 12476 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: NON_VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: true } } peers { permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: NON_VOTER last_known_addr { host: "127.3.73.252" port: 42805 } attrs { promote: true } }
I20260807 10:50:36.157106 13432 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 0c3e3fe8454b40ecafbcc66cee29bd5e (127.3.73.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: NON_VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: true } } peers { permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: NON_VOTER last_known_addr { host: "127.3.73.252" port: 42805 } attrs { promote: true } } }
I20260807 10:50:36.173832 13432 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.3.73.252:42805
I20260807 10:50:36.174507 13573 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ed8559261108476a81169c92d7e720db" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: NON_VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: true } } peers { permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: NON_VOTER last_known_addr { host: "127.3.73.252" port: 42805 } attrs { promote: true } } }
I20260807 10:50:36.175236 13368 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [term 1 LEARNER]: Refusing update from remote peer ed8559261108476a81169c92d7e720db: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 10:50:36.175359 13573 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:36.176800 13432 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [LEADER]: Connected to new peer: Peer: permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: NON_VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 10:50:36.189775 13490 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:36.193068 13299 master.cc:537] Master@127.3.73.252:42805 shutting down...
I20260807 10:50:36.194074 13368 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 0c3e3fe8454b40ecafbcc66cee29bd5e (127.3.73.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: NON_VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: true } } peers { permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: NON_VOTER last_known_addr { host: "127.3.73.252" port: 42805 } attrs { promote: true } } }
I20260807 10:50:36.202876 13413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ed8559261108476a81169c92d7e720db" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: NON_VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: true } } peers { permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: NON_VOTER last_known_addr { host: "127.3.73.252" port: 42805 } attrs { promote: true } } }
I20260807 10:50:36.203226 13413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [sys.catalog]: This master's current role is: LEARNER
I20260807 10:50:36.203962 13574 raft_consensus.cc:1050] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db: attempting to promote NON_VOTER 33fdc900cfa34b11aa90dc7c2cf3c84f to VOTER
I20260807 10:50:36.205062 13574 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: false } } peers { permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: NON_VOTER last_known_addr { host: "127.3.73.252" port: 42805 } attrs { promote: true } }
I20260807 10:50:36.208039 13367 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [term 1 LEARNER]: Refusing update from remote peer ed8559261108476a81169c92d7e720db: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 10:50:36.208926 13432 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [LEADER]: Connected to new peer: Peer: permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 10:50:36.211432 13490 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [term 1 FOLLOWER]: Refusing update from remote peer ed8559261108476a81169c92d7e720db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 10:50:36.212119 13432 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 33fdc900cfa34b11aa90dc7c2cf3c84f (127.3.73.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: false } } peers { permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: NON_VOTER last_known_addr { host: "127.3.73.252" port: 42805 } attrs { promote: true } } }
I20260807 10:50:36.215828 13299 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:50:36.216259 13299 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:50:36.216492 13299 tablet_replica.cc:330] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e: stopping tablet replica
W20260807 10:50:36.217142 12453 consensus_peers.cc:487] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db -> Peer 0c3e3fe8454b40ecafbcc66cee29bd5e (127.3.73.252:42805): Couldn't send request to peer 0c3e3fe8454b40ecafbcc66cee29bd5e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 10:50:36.218837 13368 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 33fdc900cfa34b11aa90dc7c2cf3c84f (127.3.73.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: false } } peers { permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: NON_VOTER last_known_addr { host: "127.3.73.252" port: 42805 } attrs { promote: true } } }
I20260807 10:50:36.223932 13413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ed8559261108476a81169c92d7e720db" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: false } } peers { permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: NON_VOTER last_known_addr { host: "127.3.73.252" port: 42805 } attrs { promote: true } } }
I20260807 10:50:36.224236 13413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 10:50:36.225733 13573 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ed8559261108476a81169c92d7e720db" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: false } } peers { permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: NON_VOTER last_known_addr { host: "127.3.73.252" port: 42805 } attrs { promote: true } } }
I20260807 10:50:36.226035 13573 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:36.226482 13432 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ed8559261108476a81169c92d7e720db" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: false } } peers { permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: NON_VOTER last_known_addr { host: "127.3.73.252" port: 42805 } attrs { promote: true } } }
I20260807 10:50:36.226858 13432 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:36.230526 13299 master.cc:552] Master@127.3.73.252:42805 shutdown complete.
I20260807 10:50:36.230623 13299 master_runner.cc:305] Clearing existing system tablet
I20260807 10:50:36.231046 13299 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:36.231168 13299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:36.235245 13299 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.004s
I20260807 10:50:36.237890 13583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:36.238420 13299 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:36.238718 13299 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 10:50:36.238866 13299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-2/wal
uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e"
format_stamp: "Formatted at 2026-08-07 10:50:35 on dist-test-slave-x5fp"
server_key: "6442a5fbaec5c25da3c28256bc0c7669"
server_key_iv: "37b0bc6d56f30bdfde91ad537c17709f"
server_key_version: "encryptionkey@0"
I20260807 10:50:36.245092 13299 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:50:36.249933 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 12571
I20260807 10:50:36.255630 13299 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 10:50:36.255965 13299 log.cc:1196] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 10:50:36.256938 13299 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e: Deleting consensus metadata
I20260807 10:50:36.257321 13299 master_runner.cc:315] Copying system tablet from 127.3.73.254:41969
I20260807 10:50:36.270288 13299 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e: tablet copy: Beginning tablet copy session from remote peer at address 127.3.73.254:41969
W20260807 10:50:36.283607 12456 connection.cc:535] server connection from 127.3.73.193:47689 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 10:50:36.284898 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.193:33371
--local_ip_for_outbound_sockets=127.3.73.193
--tserver_master_addrs=127.3.73.254:41969
--webserver_port=42555
--webserver_interface=127.3.73.193
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.193.keytab
--principal=oryx/127.3.73.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:35907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.3.73.254:41969,127.3.73.253:35079
--tserver_master_addrs=127.3.73.254:41969,127.3.73.253:35079,127.3.73.252:42805 with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 10:50:36.293975 13592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:36.271233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.0.0.1:56634)
0807 10:50:36.276237 (+ 5004us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:36.276270 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:36.276494 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:36.277042 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:36.277048 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:36.277804 (+ 756us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:36.278108 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:36.278123 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:36.279862 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:36.279869 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:36.281615 (+ 1746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:36.281633 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:36.282773 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:36.282780 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:36.282992 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:36.284050 (+ 1058us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:36.284078 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:36.285446 (+ 1368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:36.290389 (+ 4943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:36.290398 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:36.290405 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:36.291092 (+ 687us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:36.291612 (+ 520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:36.291618 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:36.291622 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:36.291828 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:36.292574 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:36.292583 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:36.292875 (+ 292us) client_negotiation.cc:769] Sending connection context
0807 10:50:36.293044 (+ 169us) client_negotiation.cc:240] Negotiation successful
0807 10:50:36.293444 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4828,"thread_start_us":4683,"threads_started":1}
I20260807 10:50:36.294335 13535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:36.271333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56634 (local address 127.3.73.254:41969)
0807 10:50:36.271569 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:36.271575 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:36.276437 (+ 4862us) server_negotiation.cc:407] Connection header received
0807 10:50:36.276654 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:36.276662 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:36.276837 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:36.276968 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:36.278317 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:36.279673 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:36.281800 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:36.282580 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:36.282740 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:36.285692 (+ 2952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:36.285699 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:36.285703 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:36.285752 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:36.290170 (+ 4418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:36.291280 (+ 1110us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:36.291289 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:36.291295 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:36.291470 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:36.291975 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:36.291982 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:36.291988 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:36.292409 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:36.292596 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:36.293288 (+ 692us) server_negotiation.cc:299] Negotiation successful
0807 10:50:36.293562 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260807 10:50:36.294932 12513 tablet_copy_service.cc:140] P ed8559261108476a81169c92d7e720db: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 0c3e3fe8454b40ecafbcc66cee29bd5e ({username='oryx', principal='oryx/127.3.73.252@KRBTEST.COM'} at 127.0.0.1:56634)
I20260807 10:50:36.295136 12513 tablet_copy_service.cc:161] P ed8559261108476a81169c92d7e720db: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 0c3e3fe8454b40ecafbcc66cee29bd5e at {username='oryx', principal='oryx/127.3.73.252@KRBTEST.COM'} at 127.0.0.1:56634: session id = 0c3e3fe8454b40ecafbcc66cee29bd5e-00000000000000000000000000000000
I20260807 10:50:36.299304 12513 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 10:50:36.301863 13299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 10:50:36.311182 13299 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e: tablet copy: Starting download of 0 data blocks...
I20260807 10:50:36.311627 13299 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e: tablet copy: Starting download of 1 WAL segments...
I20260807 10:50:36.315145 13299 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 10:50:36.321753 12513 tablet_copy_service.cc:342] P ed8559261108476a81169c92d7e720db: Request end of tablet copy session 0c3e3fe8454b40ecafbcc66cee29bd5e-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.3.73.252@KRBTEST.COM'} at 127.0.0.1:56634
I20260807 10:50:36.321951 12513 tablet_copy_service.cc:433] P ed8559261108476a81169c92d7e720db: ending tablet copy session 0c3e3fe8454b40ecafbcc66cee29bd5e-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 0c3e3fe8454b40ecafbcc66cee29bd5e
I20260807 10:50:36.335072 13299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:36.338018 13595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:36.342607 13596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:36.350864 13299 server_base.cc:1034] running on GCE node
W20260807 10:50:36.351307 13598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:36.351897 13299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:36.352944 13299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:36.354185 13299 hybrid_clock.cc:648] HybridClock initialized: now 1786099836354116 us; error 58 us; skew 500 ppm
I20260807 10:50:36.354517 13299 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:36 dist-test-slave-x5fp krb5kdc[12409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099836, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:36.356844 13299 init.cc:377] Logged in from keytab as oryx/127.3.73.252@KRBTEST.COM (short username oryx)
I20260807 10:50:36.358315 13299 webserver.cc:460] Webserver started at http://127.3.73.252:43961/ using document root <none> and password file <none>
I20260807 10:50:36.358650 13299 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:36.358786 13299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:36.361721 13299 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 10:50:36.365785 13606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:36.366258 13299 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260807 10:50:36.366429 13299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-2/wal
uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e"
format_stamp: "Formatted at 2026-08-07 10:50:35 on dist-test-slave-x5fp"
server_key: "6442a5fbaec5c25da3c28256bc0c7669"
server_key_iv: "37b0bc6d56f30bdfde91ad537c17709f"
server_key_version: "encryptionkey@0"
I20260807 10:50:36.366622 13299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:36.381111 13299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:36.384814 13299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:36.436218 13299 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.252:42805
I20260807 10:50:36.436281 13681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.252:42805 every 8 connection(s)
I20260807 10:50:36.437299 13299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/master-2/data/info.pb
I20260807 10:50:36.442868 13682 sys_catalog.cc:263] Verifying existing consensus state
I20260807 10:50:36.447633 13682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e: Bootstrap starting.
I20260807 10:50:36.468932 13682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 10:50:36.469556 13682 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e: Bootstrap complete.
I20260807 10:50:36.470825 13682 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: false } } peers { permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: NON_VOTER last_known_addr { host: "127.3.73.252" port: 42805 } attrs { promote: true } }
I20260807 10:50:36.471156 13682 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0c3e3fe8454b40ecafbcc66cee29bd5e, State: Initialized, Role: LEARNER
I20260807 10:50:36.471433 13682 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: false } } peers { permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: NON_VOTER last_known_addr { host: "127.3.73.252" port: 42805 } attrs { promote: true } }
I20260807 10:50:36.472281 13688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: false } } peers { permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: NON_VOTER last_known_addr { host: "127.3.73.252" port: 42805 } attrs { promote: true } } }
I20260807 10:50:36.472584 13682 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:50:36.472769 13688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [sys.catalog]: This master's current role is: LEARNER
I20260807 10:50:36.506098 13703 catalog_manager.cc:1259] Loaded cluster ID: 91f30ac1709340818f64a197b4f10202
I20260807 10:50:36.506235 13703 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e: loading cluster ID for follower catalog manager: success
I20260807 10:50:36.509255 13703 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e: acquiring CA information for follower catalog manager: success
I20260807 10:50:36.511691 13703 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 10:50:36.747953 13705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:36.738939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:42805 (local address 127.0.0.1:45732)
0807 10:50:36.739477 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:36.739499 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:36.739661 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:36.741680 (+ 2019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:36.741685 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:36.741719 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:50:36.741995 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:36.742006 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:36.743860 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:36.743866 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:36.745166 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:36.745182 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:36.746806 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:36.746812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:36.747080 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:36.747129 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 10:50:36.747269 (+ 140us) client_negotiation.cc:240] Negotiation successful
0807 10:50:36.747458 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":687,"threads_started":1}
I20260807 10:50:36.748023 13706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:36.739994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45732 (local address 127.3.73.252:42805)
0807 10:50:36.741321 (+ 1327us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:36.741326 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:36.741344 (+ 18us) server_negotiation.cc:407] Connection header received
0807 10:50:36.741409 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:36.741413 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:36.741495 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:36.741654 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:50:36.742171 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:36.743684 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:36.745482 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:36.746509 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:36.746773 (+ 264us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:36.747153 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:36.747294 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 10:50:36.747457 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1201,"thread_start_us":995,"threads_started":1}
I20260807 10:50:36.750090 13648 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260807 10:50:36.751168 13688 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed8559261108476a81169c92d7e720db. Latest consensus state: current_term: 1 leader_uuid: "ed8559261108476a81169c92d7e720db" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: false } } peers { permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: NON_VOTER last_known_addr { host: "127.3.73.252" port: 42805 } attrs { promote: true } } }
I20260807 10:50:36.751660 13688 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [sys.catalog]: This master's current role is: LEARNER
W20260807 10:50:36.846508 13594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:36.847124 13594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:36.847435 13594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:36.858543 13594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:36.858743 13594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:36.859030 13594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.193
I20260807 10:50:36.873404 13594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.193.keytab
--principal=oryx/127.3.73.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.193:33371
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.73.193
--webserver_port=42555
--tserver_master_addrs=127.3.73.254:41969,127.3.73.253:35079,127.3.73.252:42805
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.193
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:36.875007 13594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:36.876827 13594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:36.897895 13711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:36.898804 13713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:36.899643 13715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:36.901757 13594 server_base.cc:1034] running on GCE node
I20260807 10:50:36.902983 13594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:36.904455 13594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:36.905812 13594 hybrid_clock.cc:648] HybridClock initialized: now 1786099836905665 us; error 110 us; skew 500 ppm
I20260807 10:50:36.906527 13594 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:36 dist-test-slave-x5fp krb5kdc[12409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099836, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:36.912349 13594 init.cc:377] Logged in from keytab as oryx/127.3.73.193@KRBTEST.COM (short username oryx)
I20260807 10:50:36.914953 13594 webserver.cc:460] Webserver started at http://127.3.73.193:42555/ using document root <none> and password file <none>
I20260807 10:50:36.916200 13594 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:36.916432 13594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:36.924571 13594 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 10:50:36.928973 13724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:36.930927 13594 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 10:50:36.931236 13594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-0/wal
uuid: "6e1163382caf4551957d5c2ad39c678b"
format_stamp: "Formatted at 2026-08-07 10:50:33 on dist-test-slave-x5fp"
server_key: "e6f9807ac59144f776a772281b938147"
server_key_iv: "adcd6c131cd778bafa45cecf0e3238c3"
server_key_version: "encryptionkey@0"
I20260807 10:50:36.932907 13594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:36.960487 13594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:36.968048 13594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:36.968485 13594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:36.970721 13594 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:50:36.973490 13594 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:50:36.973747 13594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:36.973954 13594 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:50:36.974154 13594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:37.068634 13594 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.193:33371
I20260807 10:50:37.068969 13881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.193:33371 every 8 connection(s)
I20260807 10:50:37.070559 13594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
I20260807 10:50:37.079181 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 13594
I20260807 10:50:37.079934 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 12757
Aug 07 10:50:37 dist-test-slave-x5fp krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099836, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.252@KRBTEST.COM
Aug 07 10:50:37 dist-test-slave-x5fp krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099836, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.253@KRBTEST.COM
Aug 07 10:50:37 dist-test-slave-x5fp krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.73.194@KRBTEST.COM
Aug 07 10:50:37 dist-test-slave-x5fp krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099836, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM
I20260807 10:50:37.131253 13887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:37.076040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:42805 (local address 127.3.73.193:59793)
0807 10:50:37.086855 (+ 10815us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:37.086902 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:37.094254 (+ 7352us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:37.096338 (+ 2084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:37.096358 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:37.098791 (+ 2433us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:37.100456 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:37.100487 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:37.102528 (+ 2041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:37.102533 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:37.106800 (+ 4267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:37.106817 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:37.109053 (+ 2236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:37.109063 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:37.109609 (+ 546us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:37.110799 (+ 1190us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:37.110849 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:37.117238 (+ 6389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:37.123460 (+ 6222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:37.123473 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:37.123495 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:37.124615 (+ 1120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:37.125307 (+ 692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:37.125315 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:37.125322 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:37.125845 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:37.127013 (+ 1168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:37.127024 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:37.127857 (+ 833us) client_negotiation.cc:769] Sending connection context
0807 10:50:37.128378 (+ 521us) client_negotiation.cc:240] Negotiation successful
0807 10:50:37.129124 (+ 746us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1625,"thread_start_us":1187,"threads_started":1}
I20260807 10:50:37.132345 13890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:37.076041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:35079 (local address 127.3.73.193:37183)
0807 10:50:37.078193 (+ 2152us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:37.078245 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:37.100685 (+ 22440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:37.101526 (+ 841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:37.101531 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:37.102339 (+ 808us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:37.104402 (+ 2063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:37.104420 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:37.106560 (+ 2140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:37.106566 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:37.109294 (+ 2728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:37.109309 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:37.111425 (+ 2116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:37.111435 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:37.111992 (+ 557us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:37.113243 (+ 1251us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:37.113267 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:37.119564 (+ 6297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:37.127458 (+ 7894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:37.127465 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:37.127469 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:37.128445 (+ 976us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:37.129097 (+ 652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:37.129101 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:37.129104 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:37.129526 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:37.130584 (+ 1058us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:37.130589 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:37.130842 (+ 253us) client_negotiation.cc:769] Sending connection context
0807 10:50:37.131086 (+ 244us) client_negotiation.cc:240] Negotiation successful
0807 10:50:37.131538 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1836,"mutex_wait_us":80,"spinlock_wait_cycles":203520,"thread_start_us":1305,"threads_started":1}
I20260807 10:50:37.132973 13706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:37.075730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:59793 (local address 127.3.73.252:42805)
0807 10:50:37.075951 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:37.075958 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:37.087079 (+ 11121us) server_negotiation.cc:407] Connection header received
0807 10:50:37.094947 (+ 7868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:37.094960 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:37.095113 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:37.095223 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:37.100709 (+ 5486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:37.102283 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:37.107070 (+ 4787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:37.108213 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:37.108359 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:37.117631 (+ 9272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:37.117637 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:37.117641 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:37.117683 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:37.123136 (+ 5453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:37.124848 (+ 1712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:37.124855 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:37.124859 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:37.125104 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:37.126233 (+ 1129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:37.126239 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:37.126242 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:37.126808 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:37.127036 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:37.131960 (+ 4924us) server_negotiation.cc:299] Negotiation successful
0807 10:50:37.132359 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 10:50:37.133942 13886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:37.075055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:37183 (local address 127.3.73.253:35079)
0807 10:50:37.076882 (+ 1827us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:37.076891 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:37.078408 (+ 1517us) server_negotiation.cc:407] Connection header received
0807 10:50:37.101014 (+ 22606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:37.101022 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:37.101140 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:37.101248 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:37.104630 (+ 3382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:37.106164 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:37.109528 (+ 3364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:37.110604 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:37.110755 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:37.120284 (+ 9529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:37.120294 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:37.120299 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:37.120348 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:37.127289 (+ 6941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:37.128640 (+ 1351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:37.128660 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:37.128665 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:37.128957 (+ 292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:37.129745 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:37.129751 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:37.129766 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:37.130320 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:37.130494 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:37.132810 (+ 2316us) server_negotiation.cc:299] Negotiation successful
0807 10:50:37.133098 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1627,"thread_start_us":1461,"threads_started":1}
I20260807 10:50:37.136152 13884 heartbeater.cc:347] Connected to a master server at 127.3.73.253:35079
I20260807 10:50:37.136693 13884 heartbeater.cc:464] Registering TS with master...
I20260807 10:50:37.137499 13896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:37.082478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44954 (local address 127.3.73.194:41959)
0807 10:50:37.096754 (+ 14276us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:37.096770 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:37.098813 (+ 2043us) server_negotiation.cc:407] Connection header received
0807 10:50:37.098964 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:37.098976 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:37.099441 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:37.099654 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:37.103580 (+ 3926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:37.106342 (+ 2762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:37.110240 (+ 3898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:37.111964 (+ 1724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:37.112121 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:37.123452 (+ 11331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:37.123478 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:37.123501 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:37.123617 (+ 116us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:37.130457 (+ 6840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:37.133550 (+ 3093us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:37.133561 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:37.133564 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:37.133885 (+ 321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:37.134987 (+ 1102us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:37.134999 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:37.135002 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:37.135547 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:37.135942 (+ 395us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:37.136401 (+ 459us) server_negotiation.cc:299] Negotiation successful
0807 10:50:37.136838 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13833,"thread_start_us":14266,"threads_started":1}
I20260807 10:50:37.138587 13883 heartbeater.cc:347] Connected to a master server at 127.3.73.252:42805
I20260807 10:50:37.138635 13884 heartbeater.cc:511] Master 127.3.73.253:35079 requested a full tablet report, sending...
I20260807 10:50:37.138862 13883 heartbeater.cc:464] Registering TS with master...
I20260807 10:50:37.140157 13883 heartbeater.cc:511] Master 127.3.73.252:42805 requested a full tablet report, sending...
I20260807 10:50:37.142938 13631 ts_manager.cc:194] Registered new tserver with Master: 6e1163382caf4551957d5c2ad39c678b (127.3.73.193:33371)
I20260807 10:50:37.145514 13351 ts_manager.cc:194] Registered new tserver with Master: 6e1163382caf4551957d5c2ad39c678b (127.3.73.193:33371)
I20260807 10:50:37.147477 12931 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:37.148224 13891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:37.079198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.3.73.193:59139)
0807 10:50:37.104279 (+ 25081us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:37.104321 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:37.104709 (+ 388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:37.105619 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:37.105624 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:37.107229 (+ 1605us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:37.108508 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:37.108524 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:37.111182 (+ 2658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:37.111187 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:37.116515 (+ 5328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:37.116551 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:37.120105 (+ 3554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:37.120114 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:37.120426 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:37.121309 (+ 883us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:37.121345 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:37.128699 (+ 7354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:37.142761 (+ 14062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:37.142788 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:37.142793 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:37.143673 (+ 880us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:37.144973 (+ 1300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:37.144979 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:37.144983 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:37.145308 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:37.146548 (+ 1240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:37.146555 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:37.146926 (+ 371us) client_negotiation.cc:769] Sending connection context
0807 10:50:37.147112 (+ 186us) client_negotiation.cc:240] Negotiation successful
0807 10:50:37.147519 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24806,"mutex_wait_us":28,"thread_start_us":23990,"threads_started":1}
I20260807 10:50:37.148337 13892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:37.080925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:59139 (local address 127.3.73.254:41969)
0807 10:50:37.097751 (+ 16826us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:37.097756 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:37.104814 (+ 7058us) server_negotiation.cc:407] Connection header received
0807 10:50:37.105074 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:37.105080 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:37.105271 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:37.105395 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:37.108752 (+ 3357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:37.110940 (+ 2188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:37.116819 (+ 5879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:37.117964 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:37.118126 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:37.129059 (+ 10933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:37.129067 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:37.129072 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:37.129120 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:37.142511 (+ 13391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:37.143896 (+ 1385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:37.143903 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:37.143906 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:37.144176 (+ 270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:37.145548 (+ 1372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:37.145555 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:37.145558 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:37.146167 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:37.146422 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:37.147250 (+ 828us) server_negotiation.cc:299] Negotiation successful
0807 10:50:37.147615 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16703,"thread_start_us":16542,"threads_started":1}
I20260807 10:50:37.150043 13882 heartbeater.cc:347] Connected to a master server at 127.3.73.254:41969
I20260807 10:50:37.150308 13882 heartbeater.cc:464] Registering TS with master...
I20260807 10:50:37.151177 13882 heartbeater.cc:511] Master 127.3.73.254:41969 requested a full tablet report, sending...
I20260807 10:50:37.152913 12476 ts_manager.cc:194] Re-registered known tserver with Master: 6e1163382caf4551957d5c2ad39c678b (127.3.73.193:33371)
I20260807 10:50:37.154332 12476 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.73.193@KRBTEST.COM'} at 127.3.73.193:59139
I20260807 10:50:37.244637 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 12757
I20260807 10:50:37.270527 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.194:41959
--local_ip_for_outbound_sockets=127.3.73.194
--tserver_master_addrs=127.3.73.254:41969
--webserver_port=35231
--webserver_interface=127.3.73.194
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.194.keytab
--principal=oryx/127.3.73.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:35907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.3.73.254:41969,127.3.73.253:35079
--tserver_master_addrs=127.3.73.254:41969,127.3.73.253:35079,127.3.73.252:42805 with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 10:50:37.330436 13704 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ed8559261108476a81169c92d7e720db" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: false } } peers { permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: NON_VOTER last_known_addr { host: "127.3.73.252" port: 42805 } attrs { promote: true } } }
I20260807 10:50:37.330948 13704 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:37.332109 13704 raft_consensus.cc:1050] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db: attempting to promote NON_VOTER 0c3e3fe8454b40ecafbcc66cee29bd5e to VOTER
I20260807 10:50:37.332929 13704 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: false } } peers { permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42805 } attrs { promote: false } }
I20260807 10:50:37.335652 13368 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [term 1 FOLLOWER]: Refusing update from remote peer ed8559261108476a81169c92d7e720db: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260807 10:50:37.335760 13648 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [term 1 LEARNER]: Refusing update from remote peer ed8559261108476a81169c92d7e720db: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260807 10:50:37.336490 13902 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42805 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 10:50:37.337167 13901 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [LEADER]: Connected to new peer: Peer: permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 10:50:37.341159 13902 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 0c3e3fe8454b40ecafbcc66cee29bd5e (127.3.73.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: false } } peers { permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42805 } attrs { promote: false } } }
I20260807 10:50:37.342013 13368 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 0c3e3fe8454b40ecafbcc66cee29bd5e (127.3.73.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: false } } peers { permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42805 } attrs { promote: false } } }
I20260807 10:50:37.344813 13704 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ed8559261108476a81169c92d7e720db" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: false } } peers { permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42805 } attrs { promote: false } } }
I20260807 10:50:37.345129 13704 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:37.344856 13901 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ed8559261108476a81169c92d7e720db" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: false } } peers { permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42805 } attrs { promote: false } } }
I20260807 10:50:37.345359 13901 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:37.345999 13902 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ed8559261108476a81169c92d7e720db" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: false } } peers { permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42805 } attrs { promote: false } } }
I20260807 10:50:37.346261 13902 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:37.347322 13648 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 0c3e3fe8454b40ecafbcc66cee29bd5e (127.3.73.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: false } } peers { permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42805 } attrs { promote: false } } }
I20260807 10:50:37.351557 13909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ed8559261108476a81169c92d7e720db" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: false } } peers { permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42805 } attrs { promote: false } } }
I20260807 10:50:37.352191 13909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 10:50:37.353828 13908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ed8559261108476a81169c92d7e720db" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: false } } peers { permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42805 } attrs { promote: false } } }
I20260807 10:50:37.354353 13908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [sys.catalog]: This master's current role is: FOLLOWER
W20260807 10:50:37.678822 13900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:37.679176 13900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:37.679358 13900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:37.687083 13900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:37.687254 13900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:37.687429 13900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.194
I20260807 10:50:37.697921 13900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.194.keytab
--principal=oryx/127.3.73.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.194:41959
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.73.194
--webserver_port=35231
--tserver_master_addrs=127.3.73.254:41969,127.3.73.253:35079,127.3.73.252:42805
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.194
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:37.699499 13900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:37.701038 13900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:37.716578 13916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:37.716588 13919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:37.716825 13924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:37.717860 13900 server_base.cc:1034] running on GCE node
I20260807 10:50:37.718578 13900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:37.719624 13900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:37.720830 13900 hybrid_clock.cc:648] HybridClock initialized: now 1786099837720732 us; error 73 us; skew 500 ppm
I20260807 10:50:37.721199 13900 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:37 dist-test-slave-x5fp krb5kdc[12409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099837, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:37.729822 13900 init.cc:377] Logged in from keytab as oryx/127.3.73.194@KRBTEST.COM (short username oryx)
I20260807 10:50:37.731693 13900 webserver.cc:460] Webserver started at http://127.3.73.194:35231/ using document root <none> and password file <none>
I20260807 10:50:37.732473 13900 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:37.732602 13900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:37.739387 13900 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260807 10:50:37.742724 13935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:37.745253 13900 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 10:50:37.745451 13900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-1/wal
uuid: "29153e0ab8a84b18a1db3f02cae3aefc"
format_stamp: "Formatted at 2026-08-07 10:50:33 on dist-test-slave-x5fp"
server_key: "ff538fb60b8ab206e5d5d7fa7d223446"
server_key_iv: "5da9072f69cdc0137058a83782785c1f"
server_key_version: "encryptionkey@0"
I20260807 10:50:37.746588 13900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:37.777690 13900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:37.782377 13900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:37.782648 13900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:37.783803 13900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:50:37.785394 13900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:50:37.785490 13900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:37.785568 13900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:50:37.785624 13900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:37.824144 13900 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.194:41959
I20260807 10:50:37.824246 14113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.194:41959 every 8 connection(s)
I20260807 10:50:37.825881 13900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
I20260807 10:50:37.833470 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 13900
I20260807 10:50:37.833797 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 12947
Aug 07 10:50:37 dist-test-slave-x5fp krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.73.195@KRBTEST.COM
Aug 07 10:50:37 dist-test-slave-x5fp krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099837, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM
Aug 07 10:50:37 dist-test-slave-x5fp krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099837, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.252@KRBTEST.COM
Aug 07 10:50:37 dist-test-slave-x5fp krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099837, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.253@KRBTEST.COM
I20260807 10:50:37.878497 14124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:37.835412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34192 (local address 127.3.73.195:36361)
0807 10:50:37.836269 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:37.836284 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:37.837497 (+ 1213us) server_negotiation.cc:407] Connection header received
0807 10:50:37.838808 (+ 1311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:37.838817 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:37.839116 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:37.839369 (+ 253us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:37.842726 (+ 3357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:37.845609 (+ 2883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:37.848082 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:37.849104 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:37.849262 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:37.862087 (+ 12825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:37.862115 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:37.862140 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:37.862209 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:37.870689 (+ 8480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:37.872735 (+ 2046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:37.872749 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:37.872753 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:37.873066 (+ 313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:37.873915 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:37.873922 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:37.873927 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:37.874748 (+ 821us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:37.877291 (+ 2543us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:37.877509 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 10:50:37.877904 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":395,"threads_started":1}
I20260807 10:50:37.880498 13118 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:37.885663 14119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:37.829080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:40359 (local address 127.3.73.254:41969)
0807 10:50:37.836866 (+ 7786us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:37.836874 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:37.845858 (+ 8984us) server_negotiation.cc:407] Connection header received
0807 10:50:37.848461 (+ 2603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:37.848468 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:37.848619 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:37.848764 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:37.855003 (+ 6239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:37.857243 (+ 2240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:37.862297 (+ 5054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:37.863111 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:37.863275 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:37.871516 (+ 8241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:37.871522 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:37.871526 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:37.871599 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:37.876691 (+ 5092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:37.878715 (+ 2024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:37.878723 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:37.878727 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:37.879017 (+ 290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:37.882269 (+ 3252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:37.882276 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:37.882279 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:37.882857 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:37.883062 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:37.884602 (+ 1540us) server_negotiation.cc:299] Negotiation successful
0807 10:50:37.885046 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7643,"thread_start_us":7496,"threads_started":1}
I20260807 10:50:37.886883 14120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:37.829079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:53763 (local address 127.3.73.252:42805)
0807 10:50:37.832783 (+ 3704us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:37.832795 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:37.843131 (+ 10336us) server_negotiation.cc:407] Connection header received
0807 10:50:37.848330 (+ 5199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:37.848339 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:37.848434 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:37.849095 (+ 661us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:37.856525 (+ 7430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:37.858629 (+ 2104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:37.865341 (+ 6712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:37.866368 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:37.866526 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:37.875403 (+ 8877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:37.875413 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:37.875418 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:37.875473 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:37.881689 (+ 6216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:37.883044 (+ 1355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:37.883054 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:37.883060 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:37.883317 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:37.883940 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:37.883949 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:37.883955 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:37.884432 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:37.884710 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:37.885606 (+ 896us) server_negotiation.cc:299] Negotiation successful
0807 10:50:37.886113 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3490,"thread_start_us":3338,"threads_started":1}
I20260807 10:50:37.889314 14121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:37.829730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:42805 (local address 127.3.73.194:53763)
0807 10:50:37.842921 (+ 13191us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:37.842976 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:37.847919 (+ 4943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:37.849541 (+ 1622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:37.849546 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:37.853967 (+ 4421us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:37.856179 (+ 2212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:37.856196 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:37.861147 (+ 4951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:37.861153 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:37.864974 (+ 3821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:37.864994 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:37.867138 (+ 2144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:37.867150 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:37.867766 (+ 616us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:37.869796 (+ 2030us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:37.869828 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:37.875051 (+ 5223us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:37.881936 (+ 6885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:37.881944 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:37.881951 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:37.882823 (+ 872us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:37.883510 (+ 687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:37.883518 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:37.883523 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:37.883764 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:37.884869 (+ 1105us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:37.884889 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:37.885188 (+ 299us) client_negotiation.cc:769] Sending connection context
0807 10:50:37.887202 (+ 2014us) client_negotiation.cc:240] Negotiation successful
0807 10:50:37.887668 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12802,"mutex_wait_us":151,"spinlock_wait_cycles":73856,"thread_start_us":11152,"threads_started":1}
I20260807 10:50:37.889670 14122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:37.829730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.3.73.194:40359)
0807 10:50:37.842901 (+ 13171us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:37.842966 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:37.847919 (+ 4953us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:37.849095 (+ 1176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:37.849107 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:37.851673 (+ 2566us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:37.854423 (+ 2750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:37.854456 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:37.858911 (+ 4455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:37.858919 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:37.862052 (+ 3133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:37.862069 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:37.863527 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:37.863537 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:37.864008 (+ 471us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:37.865494 (+ 1486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:37.865547 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:37.871141 (+ 5594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:37.877203 (+ 6062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:37.877216 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:37.877248 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:37.878510 (+ 1262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:37.881615 (+ 3105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:37.881622 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:37.881625 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:37.881992 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:37.883198 (+ 1206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:37.883207 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:37.883928 (+ 721us) client_negotiation.cc:769] Sending connection context
0807 10:50:37.886091 (+ 2163us) client_negotiation.cc:240] Negotiation successful
0807 10:50:37.886933 (+ 842us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12602,"spinlock_wait_cycles":32640,"thread_start_us":12398,"threads_started":1}
I20260807 10:50:37.891491 14115 heartbeater.cc:347] Connected to a master server at 127.3.73.252:42805
I20260807 10:50:37.892081 14115 heartbeater.cc:464] Registering TS with master...
I20260807 10:50:37.892947 14114 heartbeater.cc:347] Connected to a master server at 127.3.73.254:41969
I20260807 10:50:37.893124 14114 heartbeater.cc:464] Registering TS with master...
I20260807 10:50:37.894011 14127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:37.843332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:35079 (local address 127.3.73.194:54139)
0807 10:50:37.846910 (+ 3578us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:37.846932 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:37.848363 (+ 1431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:37.850262 (+ 1899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:37.850272 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:37.852597 (+ 2325us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:37.855226 (+ 2629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:37.855244 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:37.864823 (+ 9579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:37.864834 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:37.868249 (+ 3415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:37.868269 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:37.870318 (+ 2049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:37.870329 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:37.870686 (+ 357us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:37.872659 (+ 1973us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:37.872690 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:37.881194 (+ 8504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:37.887778 (+ 6584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:37.887784 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:37.887788 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:37.888674 (+ 886us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:37.889367 (+ 693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:37.889374 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:37.889378 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:37.890394 (+ 1016us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:37.892112 (+ 1718us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:37.892117 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:37.892440 (+ 323us) client_negotiation.cc:769] Sending connection context
0807 10:50:37.892612 (+ 172us) client_negotiation.cc:240] Negotiation successful
0807 10:50:37.893240 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2958,"mutex_wait_us":1,"spinlock_wait_cycles":6272,"thread_start_us":2659,"threads_started":1}
I20260807 10:50:37.894646 14115 heartbeater.cc:511] Master 127.3.73.252:42805 requested a full tablet report, sending...
I20260807 10:50:37.894742 14114 heartbeater.cc:511] Master 127.3.73.254:41969 requested a full tablet report, sending...
I20260807 10:50:37.895406 14128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:37.843410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:54139 (local address 127.3.73.253:35079)
0807 10:50:37.845724 (+ 2314us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:37.845730 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:37.847133 (+ 1403us) server_negotiation.cc:407] Connection header received
0807 10:50:37.848513 (+ 1380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:37.848520 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:37.848627 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:37.851216 (+ 2589us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:37.855450 (+ 4234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:37.864235 (+ 8785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:37.868568 (+ 4333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:37.869319 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:37.869627 (+ 308us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:37.881624 (+ 11997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:37.881637 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:37.881642 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:37.881690 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:37.887555 (+ 5865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:37.888891 (+ 1336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:37.888901 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:37.888904 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:37.889192 (+ 288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:37.891418 (+ 2226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:37.891424 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:37.891427 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:37.891966 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:37.892140 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:37.894614 (+ 2474us) server_negotiation.cc:299] Negotiation successful
0807 10:50:37.894990 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2146,"thread_start_us":1937,"threads_started":1}
I20260807 10:50:37.897236 12476 ts_manager.cc:194] Re-registered known tserver with Master: 29153e0ab8a84b18a1db3f02cae3aefc (127.3.73.194:41959)
I20260807 10:50:37.897271 13631 ts_manager.cc:194] Registered new tserver with Master: 29153e0ab8a84b18a1db3f02cae3aefc (127.3.73.194:41959)
I20260807 10:50:37.897656 14116 heartbeater.cc:347] Connected to a master server at 127.3.73.253:35079
I20260807 10:50:37.897878 14116 heartbeater.cc:464] Registering TS with master...
I20260807 10:50:37.898576 12476 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.73.194@KRBTEST.COM'} at 127.3.73.194:40359
I20260807 10:50:37.898849 14116 heartbeater.cc:511] Master 127.3.73.253:35079 requested a full tablet report, sending...
I20260807 10:50:37.900642 13351 ts_manager.cc:194] Registered new tserver with Master: 29153e0ab8a84b18a1db3f02cae3aefc (127.3.73.194:41959)
I20260807 10:50:37.988343 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 12947
I20260807 10:50:38.015661 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.195:36361
--local_ip_for_outbound_sockets=127.3.73.195
--tserver_master_addrs=127.3.73.254:41969
--webserver_port=39451
--webserver_interface=127.3.73.195
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.195.keytab
--principal=oryx/127.3.73.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:35907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.3.73.254:41969,127.3.73.253:35079
--tserver_master_addrs=127.3.73.254:41969,127.3.73.253:35079,127.3.73.252:42805 with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 10:50:38.157588 13882 heartbeater.cc:503] Master 127.3.73.254:41969 was elected leader, sending a full tablet report...
W20260807 10:50:38.385565 14130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:38.385972 14130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:38.386148 14130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:38.395678 14130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:38.395928 14130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:38.396119 14130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.195
I20260807 10:50:38.410586 14130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.195.keytab
--principal=oryx/127.3.73.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.195:36361
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.73.195
--webserver_port=39451
--tserver_master_addrs=127.3.73.254:41969,127.3.73.253:35079,127.3.73.252:42805
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.195
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:38.412240 14130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:38.413923 14130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:38.429905 14139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:38.430001 14138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:38.429980 14141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:38.430660 14130 server_base.cc:1034] running on GCE node
I20260807 10:50:38.431521 14130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:38.432572 14130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:38.433826 14130 hybrid_clock.cc:648] HybridClock initialized: now 1786099838433759 us; error 65 us; skew 500 ppm
I20260807 10:50:38.434175 14130 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:38 dist-test-slave-x5fp krb5kdc[12409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099838, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:38.438185 14130 init.cc:377] Logged in from keytab as oryx/127.3.73.195@KRBTEST.COM (short username oryx)
I20260807 10:50:38.439972 14130 webserver.cc:460] Webserver started at http://127.3.73.195:39451/ using document root <none> and password file <none>
I20260807 10:50:38.440903 14130 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:38.441067 14130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:38.448246 14130 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 10:50:38.451390 14153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:38.453536 14130 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 10:50:38.453737 14130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-2/wal
uuid: "0ed44d75c8c349feb64451524c6536a6"
format_stamp: "Formatted at 2026-08-07 10:50:34 on dist-test-slave-x5fp"
server_key: "caadd4a334a22b953d97d39d12a31680"
server_key_iv: "68d804cb7af62aeccc7a0f3d7d8d67f2"
server_key_version: "encryptionkey@0"
I20260807 10:50:38.455260 14130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:38.477387 14130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:38.481985 14130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:38.482297 14130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:38.483461 14130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:50:38.485059 14130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:50:38.485175 14130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:38.485283 14130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:50:38.485360 14130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:38.524341 14130 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.195:36361
I20260807 10:50:38.524405 14314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.195:36361 every 8 connection(s)
I20260807 10:50:38.525913 14130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
I20260807 10:50:38.528239 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 14130
Aug 07 10:50:38 dist-test-slave-x5fp krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099838, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM
Aug 07 10:50:38 dist-test-slave-x5fp krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099838, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.252@KRBTEST.COM
Aug 07 10:50:38 dist-test-slave-x5fp krb5kdc[12409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099838, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.253@KRBTEST.COM
I20260807 10:50:38.613843 14325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:38.529936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:54019 (local address 127.3.73.252:42805)
0807 10:50:38.556691 (+ 26755us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:38.556708 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:38.556781 (+ 73us) server_negotiation.cc:407] Connection header received
0807 10:50:38.569324 (+ 12543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:38.569334 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:38.569480 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:38.569600 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:38.584594 (+ 14994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.586121 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.589419 (+ 3298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.589959 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.590115 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:38.600219 (+ 10104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:38.600229 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:38.600235 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:38.600287 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:38.605683 (+ 5396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:38.609985 (+ 4302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:38.609992 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:38.609996 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:38.610262 (+ 266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:38.610839 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:38.610845 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:38.610847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:38.611197 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:38.611385 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:38.612706 (+ 1321us) server_negotiation.cc:299] Negotiation successful
0807 10:50:38.613104 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26538,"thread_start_us":26867,"threads_started":1}
I20260807 10:50:38.615338 14326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:38.530417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:35263 (local address 127.3.73.254:41969)
0807 10:50:38.555555 (+ 25138us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:38.555567 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:38.558430 (+ 2863us) server_negotiation.cc:407] Connection header received
0807 10:50:38.567385 (+ 8955us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:38.567392 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:38.567575 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:38.567716 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:38.578588 (+ 10872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.580253 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.589291 (+ 9038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.590210 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.590419 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:38.599618 (+ 9199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:38.599625 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:38.599631 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:38.599679 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:38.605347 (+ 5668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:38.607836 (+ 2489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:38.607845 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:38.607852 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:38.608151 (+ 299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:38.610682 (+ 2531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:38.610692 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:38.610698 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:38.611326 (+ 628us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:38.611546 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:38.614552 (+ 3006us) server_negotiation.cc:299] Negotiation successful
0807 10:50:38.614800 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24911,"thread_start_us":25177,"threads_started":1}
I20260807 10:50:38.617079 14323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:38.529513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:42805 (local address 127.3.73.195:54019)
0807 10:50:38.544761 (+ 15248us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:38.544830 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:38.568924 (+ 24094us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:38.570085 (+ 1161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:38.570094 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:38.574897 (+ 4803us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:38.582861 (+ 7964us) spinlock_profiling.cc:243] Waited 4.23 ms on lock 0x61200002c628. stack: 00007f03ee60e66e 00007f03ee60e430 00007f03ed6955ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007f03ee6040fd 00007f03ee604977 00007f04029b6a7d 00007f04029b6a0e 00007f03ef73d7b7 00007f03f501203a 00007f03f5099dca 00007f03f5097f68 00007f03f50cb82f 00007f03f50cb58d
0807 10:50:38.584294 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:38.584314 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.586594 (+ 2280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.586601 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:38.589040 (+ 2439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:38.589083 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.590115 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.590121 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:38.590676 (+ 555us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:38.594204 (+ 3528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:38.594231 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:38.599920 (+ 5689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:38.608848 (+ 8928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:38.608853 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:38.608858 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:38.609781 (+ 923us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:38.610484 (+ 703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:38.610489 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:38.610494 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:38.610678 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:38.611369 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:38.611379 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:38.612101 (+ 722us) client_negotiation.cc:769] Sending connection context
0807 10:50:38.614497 (+ 2396us) client_negotiation.cc:240] Negotiation successful
0807 10:50:38.614922 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14828,"mutex_wait_us":110,"spinlock_wait_cycles":9738368,"thread_start_us":30181,"threads_started":1}
I20260807 10:50:38.617193 14321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:38.529513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.3.73.195:35263)
0807 10:50:38.558214 (+ 28701us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:38.558248 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:38.566878 (+ 8630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:38.568095 (+ 1217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:38.568109 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:38.569464 (+ 1355us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:38.578321 (+ 8857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:38.578364 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.580439 (+ 2075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.580446 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:38.589008 (+ 8562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:38.589026 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.590501 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.590510 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:38.590739 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:38.592213 (+ 1474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:38.592253 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:38.597547 (+ 5294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:38.606428 (+ 8881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:38.606442 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:38.606466 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:38.607395 (+ 929us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:38.609855 (+ 2460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:38.609860 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:38.609864 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:38.610390 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:38.611479 (+ 1089us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:38.611484 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:38.612101 (+ 617us) client_negotiation.cc:769] Sending connection context
0807 10:50:38.612564 (+ 463us) client_negotiation.cc:240] Negotiation successful
0807 10:50:38.613370 (+ 806us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28436,"mutex_wait_us":124,"spinlock_wait_cycles":62336,"thread_start_us":14483,"threads_started":1}
I20260807 10:50:38.619848 14316 heartbeater.cc:347] Connected to a master server at 127.3.73.252:42805
I20260807 10:50:38.619977 14315 heartbeater.cc:347] Connected to a master server at 127.3.73.254:41969
I20260807 10:50:38.620410 14315 heartbeater.cc:464] Registering TS with master...
I20260807 10:50:38.620410 14316 heartbeater.cc:464] Registering TS with master...
I20260807 10:50:38.621929 14315 heartbeater.cc:511] Master 127.3.73.254:41969 requested a full tablet report, sending...
I20260807 10:50:38.623567 14316 heartbeater.cc:511] Master 127.3.73.252:42805 requested a full tablet report, sending...
I20260807 10:50:38.624511 12476 ts_manager.cc:194] Re-registered known tserver with Master: 0ed44d75c8c349feb64451524c6536a6 (127.3.73.195:36361)
I20260807 10:50:38.626505 12476 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.73.195@KRBTEST.COM'} at 127.3.73.195:35263
I20260807 10:50:38.626745 13631 ts_manager.cc:194] Registered new tserver with Master: 0ed44d75c8c349feb64451524c6536a6 (127.3.73.195:36361)
I20260807 10:50:38.627080 14335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:38.565116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56636 (local address 127.3.73.254:41969)
0807 10:50:38.575119 (+ 10003us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:38.575127 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:38.578361 (+ 3234us) server_negotiation.cc:407] Connection header received
0807 10:50:38.578452 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:38.578458 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:38.578581 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:38.578703 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:38.584637 (+ 5934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.586849 (+ 2212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.597778 (+ 10929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.598932 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.599090 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:38.614154 (+ 15064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:38.614161 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:38.614166 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:38.614208 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:38.620344 (+ 6136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:38.622098 (+ 1754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:38.622104 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:38.622108 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:38.622296 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:38.624634 (+ 2338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:38.624639 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:38.624643 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:38.625187 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:38.625533 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:38.626183 (+ 650us) server_negotiation.cc:299] Negotiation successful
0807 10:50:38.626494 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9804,"thread_start_us":9572,"threads_started":1}
I20260807 10:50:38.629503 14329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:38.552141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:35079 (local address 127.3.73.195:44113)
0807 10:50:38.575375 (+ 23234us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:38.575423 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:38.579293 (+ 3870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:38.581601 (+ 2308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:38.581607 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:38.583575 (+ 1968us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:38.589402 (+ 5827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:38.589417 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.591327 (+ 1910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.591333 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:38.593926 (+ 2593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:38.593947 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.595479 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.595491 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:38.595804 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:38.599785 (+ 3981us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:38.599823 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:38.606215 (+ 6392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:38.615200 (+ 8985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:38.615206 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:38.615210 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:38.615819 (+ 609us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:38.623788 (+ 7969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:38.623794 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:38.623798 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:38.624134 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:38.627737 (+ 3603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:38.627742 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:38.628020 (+ 278us) client_negotiation.cc:769] Sending connection context
0807 10:50:38.628319 (+ 299us) client_negotiation.cc:240] Negotiation successful
0807 10:50:38.628797 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15452,"thread_start_us":8937,"threads_started":1}
I20260807 10:50:38.631001 14328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:38.545900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:44113 (local address 127.3.73.253:35079)
0807 10:50:38.561580 (+ 15680us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:38.561585 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:38.578603 (+ 17018us) server_negotiation.cc:407] Connection header received
0807 10:50:38.579481 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:38.579487 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:38.579563 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:38.579666 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:38.589617 (+ 9951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.591163 (+ 1546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.594097 (+ 2934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.595123 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.595301 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:38.606573 (+ 11272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:38.606585 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:38.606591 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:38.606644 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:38.613970 (+ 7326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:38.622488 (+ 8518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:38.622496 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:38.622500 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:38.622731 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:38.626484 (+ 3753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:38.626490 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:38.626493 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:38.626963 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:38.627179 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:38.629872 (+ 2693us) server_negotiation.cc:299] Negotiation successful
0807 10:50:38.630182 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15524,"thread_start_us":15399,"threads_started":1}
I20260807 10:50:38.631839 14340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:38.576036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43334 (local address 127.3.73.253:35079)
0807 10:50:38.578576 (+ 2540us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:38.578583 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:38.578603 (+ 20us) server_negotiation.cc:407] Connection header received
0807 10:50:38.578673 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:38.578681 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:38.578766 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:38.578909 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:38.593443 (+ 14534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.597039 (+ 3596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.599615 (+ 2576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.600634 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.600793 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:38.609050 (+ 8257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:38.609068 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:38.609078 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:38.609131 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:38.623597 (+ 14466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:38.627255 (+ 3658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:38.627262 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:38.627266 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:38.627529 (+ 263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:38.628177 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:38.628183 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:38.628186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:38.628823 (+ 637us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:38.630499 (+ 1676us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:38.630699 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 10:50:38.631057 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2376,"thread_start_us":2167,"threads_started":1}
I20260807 10:50:38.632236 14336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:38.565581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45742 (local address 127.3.73.252:42805)
0807 10:50:38.580156 (+ 14575us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:38.580164 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:38.580190 (+ 26us) server_negotiation.cc:407] Connection header received
0807 10:50:38.580260 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:38.580265 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:38.580389 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:38.580606 (+ 217us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:38.586378 (+ 5772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.588588 (+ 2210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.593183 (+ 4595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.594379 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.594555 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:38.604957 (+ 10402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:38.604973 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:38.604979 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:38.605034 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:38.618917 (+ 13883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:38.620163 (+ 1246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:38.620170 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:38.620175 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:38.620417 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:38.622098 (+ 1681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:38.622104 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:38.622108 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:38.622588 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:38.622910 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:38.624443 (+ 1533us) server_negotiation.cc:299] Negotiation successful
0807 10:50:38.624796 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14279,"thread_start_us":15720,"threads_started":1}
I20260807 10:50:38.632318 14319 heartbeater.cc:347] Connected to a master server at 127.3.73.253:35079
I20260807 10:50:38.632530 14319 heartbeater.cc:464] Registering TS with master...
I20260807 10:50:38.634027 14319 heartbeater.cc:511] Master 127.3.73.253:35079 requested a full tablet report, sending...
I20260807 10:50:38.635411 13351 ts_manager.cc:194] Registered new tserver with Master: 0ed44d75c8c349feb64451524c6536a6 (127.3.73.195:36361)
I20260807 10:50:38.634940 12476 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56636:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 10:50:38.647022 12476 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 10:50:38.718253 14366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:38.686381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34194 (local address 127.3.73.195:36361)
0807 10:50:38.704525 (+ 18144us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:38.704535 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:38.704604 (+ 69us) server_negotiation.cc:407] Connection header received
0807 10:50:38.704710 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:38.704746 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:38.705010 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:38.705256 (+ 246us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:50:38.707177 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.709045 (+ 1868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.714050 (+ 5005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.716398 (+ 2348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.716592 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:38.717038 (+ 446us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:38.717343 (+ 305us) server_negotiation.cc:299] Negotiation successful
0807 10:50:38.717564 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17834,"thread_start_us":17539,"threads_started":1}
I20260807 10:50:38.719334 14365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:38.685737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:36361 (local address 127.0.0.1:34194)
0807 10:50:38.699993 (+ 14256us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:38.700055 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:38.700269 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:38.705241 (+ 4972us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:38.705245 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:38.705273 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:50:38.706779 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:38.706798 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.709242 (+ 2444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.709248 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:38.713724 (+ 4476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:38.713744 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.716782 (+ 3038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.716792 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:38.717136 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:38.717165 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 10:50:38.717287 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 10:50:38.718025 (+ 738us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13918,"mutex_wait_us":17,"thread_start_us":16472,"threads_started":1}
I20260807 10:50:38.720966 14363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:38.684608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:41959 (local address 127.0.0.1:44964)
0807 10:50:38.701018 (+ 16410us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:38.701048 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:38.701443 (+ 395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:38.702673 (+ 1230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:38.702682 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:38.702725 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:50:38.703669 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:38.703692 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.706606 (+ 2914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.706612 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:38.714805 (+ 8193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:38.714821 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.718955 (+ 4134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.718965 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:38.719231 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:38.719284 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 10:50:38.719407 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 10:50:38.719923 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1480,"mutex_wait_us":128,"thread_start_us":959,"threads_started":1}
I20260807 10:50:38.721493 14364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:38.685591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44964 (local address 127.3.73.194:41959)
0807 10:50:38.696337 (+ 10746us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:38.696353 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:38.701209 (+ 4856us) server_negotiation.cc:407] Connection header received
0807 10:50:38.701752 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:38.701773 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:38.702308 (+ 535us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:38.702660 (+ 352us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:50:38.704099 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.706372 (+ 2273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.715076 (+ 8704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.717722 (+ 2646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.717908 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:38.718292 (+ 384us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:38.719472 (+ 1180us) server_negotiation.cc:299] Negotiation successful
0807 10:50:38.720992 (+ 1520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10440,"thread_start_us":10260,"threads_started":1}
I20260807 10:50:38.722667 14367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:38.686115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:33371 (local address 127.0.0.1:44348)
0807 10:50:38.702661 (+ 16546us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:38.702681 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:38.703217 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:38.706010 (+ 2793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:38.706015 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:38.706059 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:50:38.709306 (+ 3247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:38.709319 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.711815 (+ 2496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.711821 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:38.715412 (+ 3591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:38.715429 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.720172 (+ 4743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.720181 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:38.720474 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:38.720520 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 10:50:38.720621 (+ 101us) client_negotiation.cc:240] Negotiation successful
0807 10:50:38.720798 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16380,"mutex_wait_us":46,"thread_start_us":14284,"threads_started":1}
I20260807 10:50:38.725023 14217 tablet_service.cc:1467] Processing CreateTablet for tablet 95b0accc496142aabfde8ee3a17e05df (DEFAULT_TABLE table=test-table [id=a3a16dd8ffe641398d49431da95ac282]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:50:38.725447 14015 tablet_service.cc:1467] Processing CreateTablet for tablet 95b0accc496142aabfde8ee3a17e05df (DEFAULT_TABLE table=test-table [id=a3a16dd8ffe641398d49431da95ac282]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:50:38.726948 14015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95b0accc496142aabfde8ee3a17e05df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:38.726961 14369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:38.686726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44348 (local address 127.3.73.193:33371)
0807 10:50:38.701811 (+ 15085us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:38.701826 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:38.704853 (+ 3027us) server_negotiation.cc:407] Connection header received
0807 10:50:38.705110 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:38.705121 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:38.705385 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:38.705675 (+ 290us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:50:38.709844 (+ 4169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.711642 (+ 1798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.715621 (+ 3979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.719730 (+ 4109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.719895 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:38.720157 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:38.726051 (+ 5894us) server_negotiation.cc:299] Negotiation successful
0807 10:50:38.726321 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14809,"thread_start_us":14391,"threads_started":1}
I20260807 10:50:38.727159 14217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95b0accc496142aabfde8ee3a17e05df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:38.733140 13791 tablet_service.cc:1467] Processing CreateTablet for tablet 95b0accc496142aabfde8ee3a17e05df (DEFAULT_TABLE table=test-table [id=a3a16dd8ffe641398d49431da95ac282]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:50:38.735638 13791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95b0accc496142aabfde8ee3a17e05df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:38.748059 14370 tablet_bootstrap.cc:492] T 95b0accc496142aabfde8ee3a17e05df P 0ed44d75c8c349feb64451524c6536a6: Bootstrap starting.
I20260807 10:50:38.752615 14371 tablet_bootstrap.cc:492] T 95b0accc496142aabfde8ee3a17e05df P 29153e0ab8a84b18a1db3f02cae3aefc: Bootstrap starting.
I20260807 10:50:38.755254 14370 tablet_bootstrap.cc:654] T 95b0accc496142aabfde8ee3a17e05df P 0ed44d75c8c349feb64451524c6536a6: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:38.758901 14371 tablet_bootstrap.cc:654] T 95b0accc496142aabfde8ee3a17e05df P 29153e0ab8a84b18a1db3f02cae3aefc: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:38.759603 14370 log.cc:824] T 95b0accc496142aabfde8ee3a17e05df P 0ed44d75c8c349feb64451524c6536a6: Log is configured to *not* fsync() on all Append() calls
I20260807 10:50:38.761807 14371 log.cc:824] T 95b0accc496142aabfde8ee3a17e05df P 29153e0ab8a84b18a1db3f02cae3aefc: Log is configured to *not* fsync() on all Append() calls
I20260807 10:50:38.764698 14374 tablet_bootstrap.cc:492] T 95b0accc496142aabfde8ee3a17e05df P 6e1163382caf4551957d5c2ad39c678b: Bootstrap starting.
I20260807 10:50:38.766148 14370 tablet_bootstrap.cc:492] T 95b0accc496142aabfde8ee3a17e05df P 0ed44d75c8c349feb64451524c6536a6: No bootstrap required, opened a new log
I20260807 10:50:38.766588 14370 ts_tablet_manager.cc:1397] T 95b0accc496142aabfde8ee3a17e05df P 0ed44d75c8c349feb64451524c6536a6: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.012s
I20260807 10:50:38.771864 14374 tablet_bootstrap.cc:654] T 95b0accc496142aabfde8ee3a17e05df P 6e1163382caf4551957d5c2ad39c678b: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:38.775113 14374 log.cc:824] T 95b0accc496142aabfde8ee3a17e05df P 6e1163382caf4551957d5c2ad39c678b: Log is configured to *not* fsync() on all Append() calls
I20260807 10:50:38.780392 14371 tablet_bootstrap.cc:492] T 95b0accc496142aabfde8ee3a17e05df P 29153e0ab8a84b18a1db3f02cae3aefc: No bootstrap required, opened a new log
I20260807 10:50:38.780906 14371 ts_tablet_manager.cc:1397] T 95b0accc496142aabfde8ee3a17e05df P 29153e0ab8a84b18a1db3f02cae3aefc: Time spent bootstrapping tablet: real 0.028s user 0.010s sys 0.011s
I20260807 10:50:38.782176 14374 tablet_bootstrap.cc:492] T 95b0accc496142aabfde8ee3a17e05df P 6e1163382caf4551957d5c2ad39c678b: No bootstrap required, opened a new log
I20260807 10:50:38.782585 14374 ts_tablet_manager.cc:1397] T 95b0accc496142aabfde8ee3a17e05df P 6e1163382caf4551957d5c2ad39c678b: Time spent bootstrapping tablet: real 0.018s user 0.000s sys 0.015s
I20260807 10:50:38.786844 14370 raft_consensus.cc:348] T 95b0accc496142aabfde8ee3a17e05df P 0ed44d75c8c349feb64451524c6536a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29153e0ab8a84b18a1db3f02cae3aefc" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41959 } } peers { permanent_uuid: "6e1163382caf4551957d5c2ad39c678b" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33371 } } peers { permanent_uuid: "0ed44d75c8c349feb64451524c6536a6" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36361 } }
I20260807 10:50:38.787824 14370 raft_consensus.cc:374] T 95b0accc496142aabfde8ee3a17e05df P 0ed44d75c8c349feb64451524c6536a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:38.788046 14370 raft_consensus.cc:729] T 95b0accc496142aabfde8ee3a17e05df P 0ed44d75c8c349feb64451524c6536a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ed44d75c8c349feb64451524c6536a6, State: Initialized, Role: FOLLOWER
I20260807 10:50:38.789328 14370 consensus_queue.cc:260] T 95b0accc496142aabfde8ee3a17e05df P 0ed44d75c8c349feb64451524c6536a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29153e0ab8a84b18a1db3f02cae3aefc" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41959 } } peers { permanent_uuid: "6e1163382caf4551957d5c2ad39c678b" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33371 } } peers { permanent_uuid: "0ed44d75c8c349feb64451524c6536a6" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36361 } }
I20260807 10:50:38.791805 14315 heartbeater.cc:503] Master 127.3.73.254:41969 was elected leader, sending a full tablet report...
I20260807 10:50:38.792867 14370 ts_tablet_manager.cc:1428] T 95b0accc496142aabfde8ee3a17e05df P 0ed44d75c8c349feb64451524c6536a6: Time spent starting tablet: real 0.026s user 0.013s sys 0.014s
I20260807 10:50:38.799439 14371 raft_consensus.cc:348] T 95b0accc496142aabfde8ee3a17e05df P 29153e0ab8a84b18a1db3f02cae3aefc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29153e0ab8a84b18a1db3f02cae3aefc" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41959 } } peers { permanent_uuid: "6e1163382caf4551957d5c2ad39c678b" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33371 } } peers { permanent_uuid: "0ed44d75c8c349feb64451524c6536a6" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36361 } }
I20260807 10:50:38.800375 14371 raft_consensus.cc:374] T 95b0accc496142aabfde8ee3a17e05df P 29153e0ab8a84b18a1db3f02cae3aefc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:38.800671 14371 raft_consensus.cc:729] T 95b0accc496142aabfde8ee3a17e05df P 29153e0ab8a84b18a1db3f02cae3aefc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29153e0ab8a84b18a1db3f02cae3aefc, State: Initialized, Role: FOLLOWER
I20260807 10:50:38.801744 14371 consensus_queue.cc:260] T 95b0accc496142aabfde8ee3a17e05df P 29153e0ab8a84b18a1db3f02cae3aefc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29153e0ab8a84b18a1db3f02cae3aefc" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41959 } } peers { permanent_uuid: "6e1163382caf4551957d5c2ad39c678b" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33371 } } peers { permanent_uuid: "0ed44d75c8c349feb64451524c6536a6" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36361 } }
I20260807 10:50:38.801602 14374 raft_consensus.cc:348] T 95b0accc496142aabfde8ee3a17e05df P 6e1163382caf4551957d5c2ad39c678b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29153e0ab8a84b18a1db3f02cae3aefc" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41959 } } peers { permanent_uuid: "6e1163382caf4551957d5c2ad39c678b" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33371 } } peers { permanent_uuid: "0ed44d75c8c349feb64451524c6536a6" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36361 } }
I20260807 10:50:38.802294 14374 raft_consensus.cc:374] T 95b0accc496142aabfde8ee3a17e05df P 6e1163382caf4551957d5c2ad39c678b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:38.802498 14374 raft_consensus.cc:729] T 95b0accc496142aabfde8ee3a17e05df P 6e1163382caf4551957d5c2ad39c678b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e1163382caf4551957d5c2ad39c678b, State: Initialized, Role: FOLLOWER
I20260807 10:50:38.803505 14374 consensus_queue.cc:260] T 95b0accc496142aabfde8ee3a17e05df P 6e1163382caf4551957d5c2ad39c678b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29153e0ab8a84b18a1db3f02cae3aefc" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41959 } } peers { permanent_uuid: "6e1163382caf4551957d5c2ad39c678b" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33371 } } peers { permanent_uuid: "0ed44d75c8c349feb64451524c6536a6" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36361 } }
I20260807 10:50:38.803994 14114 heartbeater.cc:503] Master 127.3.73.254:41969 was elected leader, sending a full tablet report...
I20260807 10:50:38.804708 14371 ts_tablet_manager.cc:1428] T 95b0accc496142aabfde8ee3a17e05df P 29153e0ab8a84b18a1db3f02cae3aefc: Time spent starting tablet: real 0.024s user 0.009s sys 0.010s
I20260807 10:50:38.806530 14374 ts_tablet_manager.cc:1428] T 95b0accc496142aabfde8ee3a17e05df P 6e1163382caf4551957d5c2ad39c678b: Time spent starting tablet: real 0.024s user 0.014s sys 0.011s
W20260807 10:50:38.827831 13885 tablet.cc:2369] T 95b0accc496142aabfde8ee3a17e05df P 6e1163382caf4551957d5c2ad39c678b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 10:50:38.838795 14117 tablet.cc:2369] T 95b0accc496142aabfde8ee3a17e05df P 29153e0ab8a84b18a1db3f02cae3aefc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:50:38.976222 14380 raft_consensus.cc:482] T 95b0accc496142aabfde8ee3a17e05df P 6e1163382caf4551957d5c2ad39c678b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 10:50:38.976665 14380 raft_consensus.cc:504] T 95b0accc496142aabfde8ee3a17e05df P 6e1163382caf4551957d5c2ad39c678b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29153e0ab8a84b18a1db3f02cae3aefc" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41959 } } peers { permanent_uuid: "6e1163382caf4551957d5c2ad39c678b" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33371 } } peers { permanent_uuid: "0ed44d75c8c349feb64451524c6536a6" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36361 } }
I20260807 10:50:38.979004 14380 leader_election.cc:288] T 95b0accc496142aabfde8ee3a17e05df P 6e1163382caf4551957d5c2ad39c678b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29153e0ab8a84b18a1db3f02cae3aefc (127.3.73.194:41959), 0ed44d75c8c349feb64451524c6536a6 (127.3.73.195:36361)
I20260807 10:50:38.989236 14364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:38.980017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:49287 (local address 127.3.73.194:41959)
0807 10:50:38.980333 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:38.980343 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:38.981606 (+ 1263us) server_negotiation.cc:407] Connection header received
0807 10:50:38.981912 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:38.981920 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:38.982024 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:38.982119 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:50:38.983614 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.984580 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.987308 (+ 2728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.988195 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.988386 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:38.988469 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:38.988782 (+ 313us) server_negotiation.cc:299] Negotiation successful
0807 10:50:38.988919 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260807 10:50:38.989713 14382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:38.979204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:41959 (local address 127.3.73.193:49287)
0807 10:50:38.981435 (+ 2231us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:38.981484 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:38.981734 (+ 250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:38.982202 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:38.982229 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:38.982317 (+ 88us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:50:38.983409 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:38.983423 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.984834 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.984838 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:38.987142 (+ 2304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:38.987155 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.988430 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.988437 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:38.988584 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:38.988608 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 10:50:38.988720 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 10:50:38.988885 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1804,"mutex_wait_us":17,"thread_start_us":1223,"threads_started":1}
I20260807 10:50:38.990597 14366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:38.980080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:34885 (local address 127.3.73.195:36361)
0807 10:50:38.980370 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:38.980376 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:38.982024 (+ 1648us) server_negotiation.cc:407] Connection header received
0807 10:50:38.982450 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:38.982457 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:38.982578 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:38.982736 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:50:38.983937 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.984971 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.987860 (+ 2889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.989215 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.989472 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:38.989560 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:38.990070 (+ 510us) server_negotiation.cc:299] Negotiation successful
0807 10:50:38.990210 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260807 10:50:38.991006 14383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:38.979338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:36361 (local address 127.3.73.193:34885)
0807 10:50:38.981881 (+ 2543us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:38.981897 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:38.982187 (+ 290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:38.982762 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:38.982767 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:38.982815 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:50:38.983732 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:38.983742 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.985159 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.985163 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:38.987654 (+ 2491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:38.987668 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:38.989530 (+ 1862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:38.989537 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:38.989835 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:38.989870 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 10:50:38.990037 (+ 167us) client_negotiation.cc:240] Negotiation successful
0807 10:50:38.990176 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2409,"mutex_wait_us":21,"thread_start_us":2048,"threads_started":1}
I20260807 10:50:38.991652 14043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95b0accc496142aabfde8ee3a17e05df" candidate_uuid: "6e1163382caf4551957d5c2ad39c678b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29153e0ab8a84b18a1db3f02cae3aefc" is_pre_election: true
I20260807 10:50:38.992605 14043 raft_consensus.cc:2454] T 95b0accc496142aabfde8ee3a17e05df P 29153e0ab8a84b18a1db3f02cae3aefc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e1163382caf4551957d5c2ad39c678b in term 0.
I20260807 10:50:38.992656 14248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95b0accc496142aabfde8ee3a17e05df" candidate_uuid: "6e1163382caf4551957d5c2ad39c678b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ed44d75c8c349feb64451524c6536a6" is_pre_election: true
I20260807 10:50:38.993364 14248 raft_consensus.cc:2454] T 95b0accc496142aabfde8ee3a17e05df P 0ed44d75c8c349feb64451524c6536a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e1163382caf4551957d5c2ad39c678b in term 0.
I20260807 10:50:38.993803 13730 leader_election.cc:302] T 95b0accc496142aabfde8ee3a17e05df P 6e1163382caf4551957d5c2ad39c678b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29153e0ab8a84b18a1db3f02cae3aefc, 6e1163382caf4551957d5c2ad39c678b; no voters:
I20260807 10:50:38.994493 14380 raft_consensus.cc:2781] T 95b0accc496142aabfde8ee3a17e05df P 6e1163382caf4551957d5c2ad39c678b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 10:50:38.994817 14380 raft_consensus.cc:482] T 95b0accc496142aabfde8ee3a17e05df P 6e1163382caf4551957d5c2ad39c678b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 10:50:38.994990 14380 raft_consensus.cc:3037] T 95b0accc496142aabfde8ee3a17e05df P 6e1163382caf4551957d5c2ad39c678b [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:38.998657 14380 raft_consensus.cc:504] T 95b0accc496142aabfde8ee3a17e05df P 6e1163382caf4551957d5c2ad39c678b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29153e0ab8a84b18a1db3f02cae3aefc" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41959 } } peers { permanent_uuid: "6e1163382caf4551957d5c2ad39c678b" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33371 } } peers { permanent_uuid: "0ed44d75c8c349feb64451524c6536a6" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36361 } }
I20260807 10:50:38.999770 14380 leader_election.cc:288] T 95b0accc496142aabfde8ee3a17e05df P 6e1163382caf4551957d5c2ad39c678b [CANDIDATE]: Term 1 election: Requested vote from peers 29153e0ab8a84b18a1db3f02cae3aefc (127.3.73.194:41959), 0ed44d75c8c349feb64451524c6536a6 (127.3.73.195:36361)
I20260807 10:50:39.000411 14043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95b0accc496142aabfde8ee3a17e05df" candidate_uuid: "6e1163382caf4551957d5c2ad39c678b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29153e0ab8a84b18a1db3f02cae3aefc"
I20260807 10:50:39.000610 14248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95b0accc496142aabfde8ee3a17e05df" candidate_uuid: "6e1163382caf4551957d5c2ad39c678b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ed44d75c8c349feb64451524c6536a6"
I20260807 10:50:39.000818 14043 raft_consensus.cc:3037] T 95b0accc496142aabfde8ee3a17e05df P 29153e0ab8a84b18a1db3f02cae3aefc [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:39.000933 14248 raft_consensus.cc:3037] T 95b0accc496142aabfde8ee3a17e05df P 0ed44d75c8c349feb64451524c6536a6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:39.005257 14043 raft_consensus.cc:2454] T 95b0accc496142aabfde8ee3a17e05df P 29153e0ab8a84b18a1db3f02cae3aefc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e1163382caf4551957d5c2ad39c678b in term 1.
I20260807 10:50:39.005263 14248 raft_consensus.cc:2454] T 95b0accc496142aabfde8ee3a17e05df P 0ed44d75c8c349feb64451524c6536a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e1163382caf4551957d5c2ad39c678b in term 1.
I20260807 10:50:39.005789 13729 leader_election.cc:302] T 95b0accc496142aabfde8ee3a17e05df P 6e1163382caf4551957d5c2ad39c678b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ed44d75c8c349feb64451524c6536a6, 6e1163382caf4551957d5c2ad39c678b; no voters:
I20260807 10:50:39.006138 14380 raft_consensus.cc:2781] T 95b0accc496142aabfde8ee3a17e05df P 6e1163382caf4551957d5c2ad39c678b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:50:39.006963 14380 raft_consensus.cc:686] T 95b0accc496142aabfde8ee3a17e05df P 6e1163382caf4551957d5c2ad39c678b [term 1 LEADER]: Becoming Leader. State: Replica: 6e1163382caf4551957d5c2ad39c678b, State: Running, Role: LEADER
I20260807 10:50:39.007565 14380 consensus_queue.cc:237] T 95b0accc496142aabfde8ee3a17e05df P 6e1163382caf4551957d5c2ad39c678b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29153e0ab8a84b18a1db3f02cae3aefc" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41959 } } peers { permanent_uuid: "6e1163382caf4551957d5c2ad39c678b" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33371 } } peers { permanent_uuid: "0ed44d75c8c349feb64451524c6536a6" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36361 } }
I20260807 10:50:39.014421 12476 catalog_manager.cc:5495] T 95b0accc496142aabfde8ee3a17e05df P 6e1163382caf4551957d5c2ad39c678b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e1163382caf4551957d5c2ad39c678b (127.3.73.193). New cstate: current_term: 1 leader_uuid: "6e1163382caf4551957d5c2ad39c678b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29153e0ab8a84b18a1db3f02cae3aefc" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6e1163382caf4551957d5c2ad39c678b" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33371 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ed44d75c8c349feb64451524c6536a6" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36361 } health_report { overall_health: UNKNOWN } } }
W20260807 10:50:39.029332 14320 tablet.cc:2369] T 95b0accc496142aabfde8ee3a17e05df P 0ed44d75c8c349feb64451524c6536a6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:50:39.082799 14369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:39.075244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44362 (local address 127.3.73.193:33371)
0807 10:50:39.075556 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:39.075562 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:39.075579 (+ 17us) server_negotiation.cc:407] Connection header received
0807 10:50:39.075664 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:39.075667 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:39.075787 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:39.075954 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:50:39.076576 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:39.077832 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:39.079763 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:39.080490 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:39.080641 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:39.081431 (+ 790us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:50:39.082025 (+ 594us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:50:39.082193 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:39.082285 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 10:50:39.082376 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 10:50:39.095011 14248 raft_consensus.cc:1261] T 95b0accc496142aabfde8ee3a17e05df P 0ed44d75c8c349feb64451524c6536a6 [term 1 FOLLOWER]: Refusing update from remote peer 6e1163382caf4551957d5c2ad39c678b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:50:39.095044 14043 raft_consensus.cc:1261] T 95b0accc496142aabfde8ee3a17e05df P 29153e0ab8a84b18a1db3f02cae3aefc [term 1 FOLLOWER]: Refusing update from remote peer 6e1163382caf4551957d5c2ad39c678b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:50:39.096621 14384 consensus_queue.cc:1035] T 95b0accc496142aabfde8ee3a17e05df P 6e1163382caf4551957d5c2ad39c678b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ed44d75c8c349feb64451524c6536a6" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260807 10:50:39.097471 14380 consensus_queue.cc:1035] T 95b0accc496142aabfde8ee3a17e05df P 6e1163382caf4551957d5c2ad39c678b [LEADER]: Connected to new peer: Peer: permanent_uuid: "29153e0ab8a84b18a1db3f02cae3aefc" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:50:39.120549 14393 mvcc.cc:205] Tried to move back new op lower bound from 7315864940918902784 to 7315864940579160064. Current Snapshot: MvccSnapshot[applied={T|T < 7315864940918902784}]
I20260807 10:50:39.147974 12474 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:50:39.148301 12474 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56636:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:50:39.156371 12474 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ed8559261108476a81169c92d7e720db: Sending DeleteTablet for 3 replicas of tablet 95b0accc496142aabfde8ee3a17e05df
I20260807 10:50:39.158205 13791 tablet_service.cc:1514] Processing DeleteTablet for tablet 95b0accc496142aabfde8ee3a17e05df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:50:39 UTC) from {username='oryx', principal='oryx/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:44348
I20260807 10:50:39.159374 14217 tablet_service.cc:1514] Processing DeleteTablet for tablet 95b0accc496142aabfde8ee3a17e05df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:50:39 UTC) from {username='oryx', principal='oryx/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:34194
I20260807 10:50:39.159283 14015 tablet_service.cc:1514] Processing DeleteTablet for tablet 95b0accc496142aabfde8ee3a17e05df with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:50:39 UTC) from {username='oryx', principal='oryx/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:44964
I20260807 10:50:39.160795 14403 tablet_replica.cc:330] T 95b0accc496142aabfde8ee3a17e05df P 6e1163382caf4551957d5c2ad39c678b: stopping tablet replica
I20260807 10:50:39.161809 14403 raft_consensus.cc:2229] T 95b0accc496142aabfde8ee3a17e05df P 6e1163382caf4551957d5c2ad39c678b [term 1 LEADER]: Raft consensus shutting down.
I20260807 10:50:39.163442 14403 raft_consensus.cc:2258] T 95b0accc496142aabfde8ee3a17e05df P 6e1163382caf4551957d5c2ad39c678b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:50:39.166668 14403 ts_tablet_manager.cc:1905] T 95b0accc496142aabfde8ee3a17e05df P 6e1163382caf4551957d5c2ad39c678b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:50:39.173743 14404 tablet_replica.cc:330] T 95b0accc496142aabfde8ee3a17e05df P 29153e0ab8a84b18a1db3f02cae3aefc: stopping tablet replica
I20260807 10:50:39.174680 14404 raft_consensus.cc:2229] T 95b0accc496142aabfde8ee3a17e05df P 29153e0ab8a84b18a1db3f02cae3aefc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:50:39.175561 14404 raft_consensus.cc:2258] T 95b0accc496142aabfde8ee3a17e05df P 29153e0ab8a84b18a1db3f02cae3aefc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:50:39.179764 14404 ts_tablet_manager.cc:1905] T 95b0accc496142aabfde8ee3a17e05df P 29153e0ab8a84b18a1db3f02cae3aefc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:50:39.181135 14403 ts_tablet_manager.cc:1918] T 95b0accc496142aabfde8ee3a17e05df P 6e1163382caf4551957d5c2ad39c678b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:50:39.181566 14403 log.cc:1196] T 95b0accc496142aabfde8ee3a17e05df P 6e1163382caf4551957d5c2ad39c678b: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-0/wal/wals/95b0accc496142aabfde8ee3a17e05df
I20260807 10:50:39.183115 14403 ts_tablet_manager.cc:1939] T 95b0accc496142aabfde8ee3a17e05df P 6e1163382caf4551957d5c2ad39c678b: Deleting consensus metadata
I20260807 10:50:39.186517 12457 catalog_manager.cc:4841] TS 6e1163382caf4551957d5c2ad39c678b (127.3.73.193:33371): tablet 95b0accc496142aabfde8ee3a17e05df (table test-table [id=a3a16dd8ffe641398d49431da95ac282]) successfully deleted
I20260807 10:50:39.190524 14404 ts_tablet_manager.cc:1918] T 95b0accc496142aabfde8ee3a17e05df P 29153e0ab8a84b18a1db3f02cae3aefc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:50:39.190908 14404 log.cc:1196] T 95b0accc496142aabfde8ee3a17e05df P 29153e0ab8a84b18a1db3f02cae3aefc: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-1/wal/wals/95b0accc496142aabfde8ee3a17e05df
I20260807 10:50:39.191362 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 12429
I20260807 10:50:39.192021 14404 ts_tablet_manager.cc:1939] T 95b0accc496142aabfde8ee3a17e05df P 29153e0ab8a84b18a1db3f02cae3aefc: Deleting consensus metadata
I20260807 10:50:39.193264 12526 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:39.195330 12457 catalog_manager.cc:4841] TS 29153e0ab8a84b18a1db3f02cae3aefc (127.3.73.194:41959): tablet 95b0accc496142aabfde8ee3a17e05df (table test-table [id=a3a16dd8ffe641398d49431da95ac282]) successfully deleted
I20260807 10:50:39.197238 14406 tablet_replica.cc:330] T 95b0accc496142aabfde8ee3a17e05df P 0ed44d75c8c349feb64451524c6536a6: stopping tablet replica
I20260807 10:50:39.198069 14406 raft_consensus.cc:2229] T 95b0accc496142aabfde8ee3a17e05df P 0ed44d75c8c349feb64451524c6536a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:50:39.198894 14406 raft_consensus.cc:2258] T 95b0accc496142aabfde8ee3a17e05df P 0ed44d75c8c349feb64451524c6536a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:50:39.209131 14406 ts_tablet_manager.cc:1905] T 95b0accc496142aabfde8ee3a17e05df P 0ed44d75c8c349feb64451524c6536a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:50:39.221621 14406 ts_tablet_manager.cc:1918] T 95b0accc496142aabfde8ee3a17e05df P 0ed44d75c8c349feb64451524c6536a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:50:39.221992 14406 log.cc:1196] T 95b0accc496142aabfde8ee3a17e05df P 0ed44d75c8c349feb64451524c6536a6: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-2/wal/wals/95b0accc496142aabfde8ee3a17e05df
I20260807 10:50:39.223155 14406 ts_tablet_manager.cc:1939] T 95b0accc496142aabfde8ee3a17e05df P 0ed44d75c8c349feb64451524c6536a6: Deleting consensus metadata
I20260807 10:50:39.238651 14408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:39.164341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43350 (local address 127.3.73.253:35079)
0807 10:50:39.168689 (+ 4348us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:39.168696 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:39.197064 (+ 28368us) server_negotiation.cc:407] Connection header received
0807 10:50:39.198242 (+ 1178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:39.198249 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:39.198419 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:39.205645 (+ 7226us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:39.213298 (+ 7653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:39.215286 (+ 1988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:39.219220 (+ 3934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:39.220191 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:39.220349 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:39.227082 (+ 6733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:39.227092 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:39.227097 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:39.227177 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:39.233873 (+ 6696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:39.235138 (+ 1265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:39.235144 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:39.235149 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:39.235429 (+ 280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:39.236087 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:39.236092 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:39.236095 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:39.236777 (+ 682us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:39.237250 (+ 473us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:39.237709 (+ 459us) server_negotiation.cc:299] Negotiation successful
0807 10:50:39.238038 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3615,"thread_start_us":3438,"threads_started":1}
I20260807 10:50:39.267298 14409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:39.190018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45756 (local address 127.3.73.252:42805)
0807 10:50:39.217055 (+ 27037us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:39.217062 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:39.238977 (+ 21915us) server_negotiation.cc:407] Connection header received
0807 10:50:39.239416 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:39.239423 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:39.239577 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:39.239908 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:39.243883 (+ 3975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:39.245402 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:39.248326 (+ 2924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:39.249356 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:39.249535 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:39.255793 (+ 6258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:39.255801 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:39.255806 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:39.255880 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:39.261890 (+ 6010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:39.263149 (+ 1259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:39.263156 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:39.263160 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:39.263434 (+ 274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:39.264231 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:39.264237 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:39.264240 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:39.264774 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:39.265163 (+ 389us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:39.266407 (+ 1244us) server_negotiation.cc:299] Negotiation successful
0807 10:50:39.266706 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26837,"thread_start_us":26584,"threads_started":1}
I20260807 10:50:39.358672 12456 catalog_manager.cc:4841] TS 0ed44d75c8c349feb64451524c6536a6 (127.3.73.195:36361): tablet 95b0accc496142aabfde8ee3a17e05df (table test-table [id=a3a16dd8ffe641398d49431da95ac282]) successfully deleted
I20260807 10:50:39.360533 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 12429
I20260807 10:50:39.428694 14409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:39.409471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45760 (local address 127.3.73.252:42805)
0807 10:50:39.409689 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:39.409696 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:39.410848 (+ 1152us) server_negotiation.cc:407] Connection header received
0807 10:50:39.411131 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:39.411138 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:39.411238 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:39.411329 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:39.413993 (+ 2664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:39.415266 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:39.416982 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:39.417733 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:39.417875 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:39.421385 (+ 3510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:39.421391 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:39.421395 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:39.421436 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:39.424977 (+ 3541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:39.426097 (+ 1120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:39.426104 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:39.426109 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:39.426330 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:39.426840 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:39.426849 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:39.426853 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:39.427287 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:39.427604 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:39.428020 (+ 416us) server_negotiation.cc:299] Negotiation successful
0807 10:50:39.428270 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 10:50:39.430997 14408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:39.410632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43364 (local address 127.3.73.253:35079)
0807 10:50:39.410886 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:39.410896 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:39.412302 (+ 1406us) server_negotiation.cc:407] Connection header received
0807 10:50:39.412853 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:39.412858 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:39.412931 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:39.413051 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:39.415169 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:39.416294 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:39.418084 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:39.418708 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:39.418866 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:39.423313 (+ 4447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:39.423323 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:39.423332 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:39.423386 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:39.426992 (+ 3606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:39.427837 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:39.427846 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:39.427851 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:39.428028 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:39.428630 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:39.428636 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:39.428640 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:39.429595 (+ 955us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:39.429870 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:39.430380 (+ 510us) server_negotiation.cc:299] Negotiation successful
0807 10:50:39.430627 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 10:50:40.129187 14424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:40.128315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.3.73.195:33559)
0807 10:50:40.128877 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:40.129007 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":290,"threads_started":1}
W20260807 10:50:40.129688 14315 heartbeater.cc:644] Failed to heartbeat to 127.3.73.254:41969 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111)
I20260807 10:50:40.134117 14427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:40.132896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.3.73.193:39043)
0807 10:50:40.133693 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:40.133846 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":426,"threads_started":1}
W20260807 10:50:40.134603 13882 heartbeater.cc:644] Failed to heartbeat to 127.3.73.254:41969 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111)
I20260807 10:50:40.140774 14428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:40.139947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.3.73.194:40005)
0807 10:50:40.140437 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:40.140551 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":217,"threads_started":1}
W20260807 10:50:40.141206 14114 heartbeater.cc:644] Failed to heartbeat to 127.3.73.254:41969 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111)
I20260807 10:50:40.657512 14429 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ed8559261108476a81169c92d7e720db)
I20260807 10:50:40.657979 14430 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ed8559261108476a81169c92d7e720db)
I20260807 10:50:40.657878 14429 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: false } } peers { permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42805 } attrs { promote: false } }
I20260807 10:50:40.658234 14430 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: false } } peers { permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42805 } attrs { promote: false } }
I20260807 10:50:40.659598 14429 leader_election.cc:288] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ed8559261108476a81169c92d7e720db (127.3.73.254:41969), 33fdc900cfa34b11aa90dc7c2cf3c84f (127.3.73.253:35079)
I20260807 10:50:40.665766 14430 leader_election.cc:288] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ed8559261108476a81169c92d7e720db (127.3.73.254:41969), 0c3e3fe8454b40ecafbcc66cee29bd5e (127.3.73.252:42805)
I20260807 10:50:40.671459 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:40.659781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.0.0.1:51034)
0807 10:50:40.670738 (+ 10957us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:40.671036 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10752,"thread_start_us":10502,"threads_started":1}
W20260807 10:50:40.675225 13611 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111)
I20260807 10:50:40.675791 14434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:40.659794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.0.0.1:51038)
0807 10:50:40.675201 (+ 15407us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:40.675441 (+ 240us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15217,"threads_started":1}
W20260807 10:50:40.676563 13332 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111)
I20260807 10:50:40.685519 14436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:40.684799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.0.0.1:51058)
0807 10:50:40.685018 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:40.685228 (+ 210us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 10:50:40.685849 13611 leader_election.cc:334] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ed8559261108476a81169c92d7e720db (127.3.73.254:41969): Network error: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111)
I20260807 10:50:40.687045 14435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:40.660198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42946 (local address 127.3.73.252:42805)
0807 10:50:40.675626 (+ 15428us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:40.675633 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:40.676547 (+ 914us) server_negotiation.cc:407] Connection header received
0807 10:50:40.676639 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:40.676646 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:40.676774 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:40.677575 (+ 801us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:50:40.677970 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:40.679813 (+ 1843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:40.683542 (+ 3729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:40.685096 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:40.685265 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:40.685379 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:40.686463 (+ 1084us) server_negotiation.cc:299] Negotiation successful
0807 10:50:40.686645 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15289,"thread_start_us":15433,"threads_started":1}
I20260807 10:50:40.687161 14434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:40.659839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:42805 (local address 127.0.0.1:42946)
0807 10:50:40.676219 (+ 16380us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:40.676239 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:40.676412 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:40.676956 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:40.676961 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:40.676990 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:50:40.677363 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:40.677383 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:40.680060 (+ 2677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:40.680070 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:40.683263 (+ 3193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:40.683296 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:40.685289 (+ 1993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:40.685296 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:40.685543 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:40.685588 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 10:50:40.685709 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 10:50:40.685957 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16270,"thread_start_us":17097,"threads_started":1}
I20260807 10:50:40.687487 14432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:40.686076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.0.0.1:51066)
0807 10:50:40.686332 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:40.686676 (+ 344us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 10:50:40.687779 13332 leader_election.cc:334] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ed8559261108476a81169c92d7e720db (127.3.73.254:41969): Network error: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111)
I20260807 10:50:40.688148 13648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" is_pre_election: true
I20260807 10:50:40.688534 13648 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33fdc900cfa34b11aa90dc7c2cf3c84f in term 1.
I20260807 10:50:40.689148 13327 leader_election.cc:302] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0c3e3fe8454b40ecafbcc66cee29bd5e, 33fdc900cfa34b11aa90dc7c2cf3c84f; no voters: ed8559261108476a81169c92d7e720db
I20260807 10:50:40.689627 14430 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 10:50:40.689839 14430 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [term 1 FOLLOWER]: Starting leader election (detected failure of leader ed8559261108476a81169c92d7e720db)
I20260807 10:50:40.689982 14430 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [term 1 FOLLOWER]: Advancing to term 2
I20260807 10:50:40.692478 14430 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: false } } peers { permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42805 } attrs { promote: false } }
I20260807 10:50:40.693320 14430 leader_election.cc:288] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [CANDIDATE]: Term 2 election: Requested vote from peers ed8559261108476a81169c92d7e720db (127.3.73.254:41969), 0c3e3fe8454b40ecafbcc66cee29bd5e (127.3.73.252:42805)
I20260807 10:50:40.693835 13648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e"
I20260807 10:50:40.694053 14432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:40.693633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.0.0.1:51078)
0807 10:50:40.693789 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:40.693883 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 10:50:40.694231 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:40.671054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:35079 (local address 127.0.0.1:43174)
0807 10:50:40.673475 (+ 2421us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:40.673507 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:40.673737 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:40.688019 (+ 14282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:40.688025 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:40.688066 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:50:40.688308 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:40.688317 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:40.690362 (+ 2045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:40.690375 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:40.691736 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:40.691749 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:40.693260 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:40.693267 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:40.693490 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:40.693536 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 10:50:40.693645 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 10:50:40.693829 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2274,"thread_start_us":3694,"threads_started":1}
I20260807 10:50:40.694296 13648 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [term 1 FOLLOWER]: Advancing to term 2
I20260807 10:50:40.694424 14437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:40.671033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43174 (local address 127.3.73.253:35079)
0807 10:50:40.687620 (+ 16587us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:40.687629 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:40.687652 (+ 23us) server_negotiation.cc:407] Connection header received
0807 10:50:40.687717 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:40.687725 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:40.687825 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:40.688497 (+ 672us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:50:40.688782 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:40.690197 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:40.691931 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:40.693059 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:40.693222 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:40.693365 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:40.693700 (+ 335us) server_negotiation.cc:299] Negotiation successful
0807 10:50:40.693861 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16444,"thread_start_us":13496,"threads_started":1}
I20260807 10:50:40.695297 13368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" is_pre_election: true
I20260807 10:50:40.695590 14432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:40.695066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.0.0.1:51082)
0807 10:50:40.695205 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:40.695275 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 10:50:40.695791 13368 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0c3e3fe8454b40ecafbcc66cee29bd5e in current term 2: Already voted for candidate 33fdc900cfa34b11aa90dc7c2cf3c84f in this term.
W20260807 10:50:40.695806 13332 leader_election.cc:334] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ed8559261108476a81169c92d7e720db (127.3.73.254:41969): Network error: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111)
I20260807 10:50:40.696496 13611 leader_election.cc:302] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0c3e3fe8454b40ecafbcc66cee29bd5e; no voters: 33fdc900cfa34b11aa90dc7c2cf3c84f, ed8559261108476a81169c92d7e720db
I20260807 10:50:40.697701 13648 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33fdc900cfa34b11aa90dc7c2cf3c84f in term 2.
I20260807 10:50:40.697896 14429 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 10:50:40.698243 13327 leader_election.cc:302] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0c3e3fe8454b40ecafbcc66cee29bd5e, 33fdc900cfa34b11aa90dc7c2cf3c84f; no voters: ed8559261108476a81169c92d7e720db
I20260807 10:50:40.698639 14430 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [term 2 FOLLOWER]: Leader election won for term 2
I20260807 10:50:40.699127 14430 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [term 2 LEADER]: Becoming Leader. State: Replica: 33fdc900cfa34b11aa90dc7c2cf3c84f, State: Running, Role: LEADER
I20260807 10:50:40.699635 14430 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: false } } peers { permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42805 } attrs { promote: false } }
I20260807 10:50:40.701721 14440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33fdc900cfa34b11aa90dc7c2cf3c84f. Latest consensus state: current_term: 2 leader_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: false } } peers { permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42805 } attrs { promote: false } } }
I20260807 10:50:40.702104 14440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:40.702924 14445 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:50:40.706954 14445 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:50:40.707788 14445 catalog_manager.cc:1259] Loaded cluster ID: 91f30ac1709340818f64a197b4f10202
I20260807 10:50:40.707907 14445 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:50:40.709136 14445 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:50:40.710040 14445 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f: Loaded TSK: 0
I20260807 10:50:40.711028 14445 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 10:50:40.746016 13350 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43364:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 10:50:40.754022 13350 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 10:50:40.758899 14432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:40.758407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.0.0.1:51102)
0807 10:50:40.758588 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:40.758741 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 10:50:40.759061 13648 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [term 2 FOLLOWER]: Refusing update from remote peer 33fdc900cfa34b11aa90dc7c2cf3c84f: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260807 10:50:40.760171 14432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:40.759644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.0.0.1:51116)
0807 10:50:40.759825 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:40.759907 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 10:50:40.760073 14440 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42805 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260807 10:50:40.760464 13332 consensus_peers.cc:487] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f -> Peer ed8559261108476a81169c92d7e720db (127.3.73.254:41969): Couldn't send request to peer ed8559261108476a81169c92d7e720db. Status: Network error: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 10:50:40.763495 14429 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33fdc900cfa34b11aa90dc7c2cf3c84f. Latest consensus state: current_term: 2 leader_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: false } } peers { permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42805 } attrs { promote: false } } }
I20260807 10:50:40.763854 14429 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 10:50:40.765605 14440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: false } } peers { permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42805 } attrs { promote: false } } }
I20260807 10:50:40.765949 14440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:40.766376 14429 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ed8559261108476a81169c92d7e720db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41969 } } peers { permanent_uuid: "33fdc900cfa34b11aa90dc7c2cf3c84f" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 35079 } attrs { promote: false } } peers { permanent_uuid: "0c3e3fe8454b40ecafbcc66cee29bd5e" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42805 } attrs { promote: false } } }
I20260807 10:50:40.766677 14429 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c3e3fe8454b40ecafbcc66cee29bd5e [sys.catalog]: This master's current role is: FOLLOWER
I20260807 10:50:40.784332 14432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:40.775137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:36361 (local address 127.0.0.1:42200)
0807 10:50:40.775318 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:40.775339 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:40.775483 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:40.776123 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:40.776128 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:40.776157 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:50:40.776517 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:40.776535 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:40.779581 (+ 3046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:40.779588 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:40.781736 (+ 2148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:40.781748 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:40.783428 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:40.783435 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:40.783658 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:40.783697 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 10:50:40.783755 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 10:50:40.783949 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":119}
I20260807 10:50:40.784358 14434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:40.775985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:33371 (local address 127.0.0.1:42504)
0807 10:50:40.776820 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:40.776840 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:40.776953 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:40.777986 (+ 1033us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:40.777989 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:40.778010 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:50:40.778222 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:40.778236 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:40.779667 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:40.779673 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:40.781875 (+ 2202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:40.781899 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:40.783502 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:40.783508 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:40.783720 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:40.783770 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 10:50:40.783855 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 10:50:40.784019 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"mutex_wait_us":62}
I20260807 10:50:40.784806 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:40.776806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35690 (local address 127.3.73.194:41959)
0807 10:50:40.778560 (+ 1754us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:40.778566 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:40.778585 (+ 19us) server_negotiation.cc:407] Connection header received
0807 10:50:40.778659 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:40.778664 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:40.778736 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:40.778841 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:50:40.779468 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:40.780572 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:40.782780 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:40.783697 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:40.783844 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:40.783963 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:40.784299 (+ 336us) server_negotiation.cc:299] Negotiation successful
0807 10:50:40.784471 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1633,"thread_start_us":1528,"threads_started":1}
I20260807 10:50:40.784881 14458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:40.776297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:41959 (local address 127.0.0.1:35690)
0807 10:50:40.777430 (+ 1133us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:40.777444 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:40.777527 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:40.778880 (+ 1353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:40.778886 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:40.778914 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:50:40.779292 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:40.779303 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:40.780771 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:40.780779 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:40.782612 (+ 1833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:40.782624 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:40.783923 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:40.783928 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:40.784114 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:40.784150 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 10:50:40.784244 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 10:50:40.784407 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1018,"mutex_wait_us":22,"thread_start_us":789,"threads_started":1}
I20260807 10:50:40.785493 14457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:40.776208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42504 (local address 127.3.73.193:33371)
0807 10:50:40.776803 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:40.776810 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:40.776895 (+ 85us) server_negotiation.cc:407] Connection header received
0807 10:50:40.777724 (+ 829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:40.777733 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:40.777842 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:40.777978 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:50:40.778412 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:40.779540 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:40.782053 (+ 2513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:40.783335 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:40.783467 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:40.783602 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:40.784921 (+ 1319us) server_negotiation.cc:299] Negotiation successful
0807 10:50:40.785099 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":294,"threads_started":1}
I20260807 10:50:40.786049 14015 tablet_service.cc:1467] Processing CreateTablet for tablet 09e2bd0363354c3e9bc7d562a05e960d (DEFAULT_TABLE table=test-table [id=3120f512d26446ac90b188ac5564f904]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:50:40.786266 13791 tablet_service.cc:1467] Processing CreateTablet for tablet 09e2bd0363354c3e9bc7d562a05e960d (DEFAULT_TABLE table=test-table [id=3120f512d26446ac90b188ac5564f904]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:50:40.786454 14456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:40.775278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42200 (local address 127.3.73.195:36361)
0807 10:50:40.775774 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:40.775778 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:40.775794 (+ 16us) server_negotiation.cc:407] Connection header received
0807 10:50:40.775852 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:40.775856 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:40.775954 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:40.776079 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:50:40.778516 (+ 2437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:40.779443 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:40.781897 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:40.783158 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:40.785718 (+ 2560us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:40.785838 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:40.785907 (+ 69us) server_negotiation.cc:299] Negotiation successful
0807 10:50:40.786047 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":263,"threads_started":1}
I20260807 10:50:40.786645 14015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09e2bd0363354c3e9bc7d562a05e960d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:40.786679 13791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09e2bd0363354c3e9bc7d562a05e960d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:40.787233 14217 tablet_service.cc:1467] Processing CreateTablet for tablet 09e2bd0363354c3e9bc7d562a05e960d (DEFAULT_TABLE table=test-table [id=3120f512d26446ac90b188ac5564f904]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:50:40.787665 14217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09e2bd0363354c3e9bc7d562a05e960d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:40.791697 14460 tablet_bootstrap.cc:492] T 09e2bd0363354c3e9bc7d562a05e960d P 6e1163382caf4551957d5c2ad39c678b: Bootstrap starting.
I20260807 10:50:40.793562 14461 tablet_bootstrap.cc:492] T 09e2bd0363354c3e9bc7d562a05e960d P 29153e0ab8a84b18a1db3f02cae3aefc: Bootstrap starting.
I20260807 10:50:40.793753 14460 tablet_bootstrap.cc:654] T 09e2bd0363354c3e9bc7d562a05e960d P 6e1163382caf4551957d5c2ad39c678b: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:40.795048 14462 tablet_bootstrap.cc:492] T 09e2bd0363354c3e9bc7d562a05e960d P 0ed44d75c8c349feb64451524c6536a6: Bootstrap starting.
I20260807 10:50:40.795624 14461 tablet_bootstrap.cc:654] T 09e2bd0363354c3e9bc7d562a05e960d P 29153e0ab8a84b18a1db3f02cae3aefc: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:40.796841 14460 tablet_bootstrap.cc:492] T 09e2bd0363354c3e9bc7d562a05e960d P 6e1163382caf4551957d5c2ad39c678b: No bootstrap required, opened a new log
I20260807 10:50:40.796869 14462 tablet_bootstrap.cc:654] T 09e2bd0363354c3e9bc7d562a05e960d P 0ed44d75c8c349feb64451524c6536a6: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:40.797024 14460 ts_tablet_manager.cc:1397] T 09e2bd0363354c3e9bc7d562a05e960d P 6e1163382caf4551957d5c2ad39c678b: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.004s
I20260807 10:50:40.798054 14461 tablet_bootstrap.cc:492] T 09e2bd0363354c3e9bc7d562a05e960d P 29153e0ab8a84b18a1db3f02cae3aefc: No bootstrap required, opened a new log
I20260807 10:50:40.797963 14460 raft_consensus.cc:348] T 09e2bd0363354c3e9bc7d562a05e960d P 6e1163382caf4551957d5c2ad39c678b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed44d75c8c349feb64451524c6536a6" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36361 } } peers { permanent_uuid: "6e1163382caf4551957d5c2ad39c678b" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33371 } } peers { permanent_uuid: "29153e0ab8a84b18a1db3f02cae3aefc" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41959 } }
I20260807 10:50:40.798256 14460 raft_consensus.cc:374] T 09e2bd0363354c3e9bc7d562a05e960d P 6e1163382caf4551957d5c2ad39c678b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:40.798357 14461 ts_tablet_manager.cc:1397] T 09e2bd0363354c3e9bc7d562a05e960d P 29153e0ab8a84b18a1db3f02cae3aefc: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 10:50:40.798362 14460 raft_consensus.cc:729] T 09e2bd0363354c3e9bc7d562a05e960d P 6e1163382caf4551957d5c2ad39c678b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e1163382caf4551957d5c2ad39c678b, State: Initialized, Role: FOLLOWER
I20260807 10:50:40.798643 14460 consensus_queue.cc:260] T 09e2bd0363354c3e9bc7d562a05e960d P 6e1163382caf4551957d5c2ad39c678b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed44d75c8c349feb64451524c6536a6" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36361 } } peers { permanent_uuid: "6e1163382caf4551957d5c2ad39c678b" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33371 } } peers { permanent_uuid: "29153e0ab8a84b18a1db3f02cae3aefc" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41959 } }
I20260807 10:50:40.799350 14461 raft_consensus.cc:348] T 09e2bd0363354c3e9bc7d562a05e960d P 29153e0ab8a84b18a1db3f02cae3aefc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed44d75c8c349feb64451524c6536a6" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36361 } } peers { permanent_uuid: "6e1163382caf4551957d5c2ad39c678b" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33371 } } peers { permanent_uuid: "29153e0ab8a84b18a1db3f02cae3aefc" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41959 } }
I20260807 10:50:40.799613 14461 raft_consensus.cc:374] T 09e2bd0363354c3e9bc7d562a05e960d P 29153e0ab8a84b18a1db3f02cae3aefc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:40.799682 14460 ts_tablet_manager.cc:1428] T 09e2bd0363354c3e9bc7d562a05e960d P 6e1163382caf4551957d5c2ad39c678b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 10:50:40.799713 14461 raft_consensus.cc:729] T 09e2bd0363354c3e9bc7d562a05e960d P 29153e0ab8a84b18a1db3f02cae3aefc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29153e0ab8a84b18a1db3f02cae3aefc, State: Initialized, Role: FOLLOWER
I20260807 10:50:40.800022 14461 consensus_queue.cc:260] T 09e2bd0363354c3e9bc7d562a05e960d P 29153e0ab8a84b18a1db3f02cae3aefc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed44d75c8c349feb64451524c6536a6" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36361 } } peers { permanent_uuid: "6e1163382caf4551957d5c2ad39c678b" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33371 } } peers { permanent_uuid: "29153e0ab8a84b18a1db3f02cae3aefc" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41959 } }
I20260807 10:50:40.803532 14461 ts_tablet_manager.cc:1428] T 09e2bd0363354c3e9bc7d562a05e960d P 29153e0ab8a84b18a1db3f02cae3aefc: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 10:50:40.806437 14462 tablet_bootstrap.cc:492] T 09e2bd0363354c3e9bc7d562a05e960d P 0ed44d75c8c349feb64451524c6536a6: No bootstrap required, opened a new log
I20260807 10:50:40.806643 14462 ts_tablet_manager.cc:1397] T 09e2bd0363354c3e9bc7d562a05e960d P 0ed44d75c8c349feb64451524c6536a6: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.009s
I20260807 10:50:40.807765 14462 raft_consensus.cc:348] T 09e2bd0363354c3e9bc7d562a05e960d P 0ed44d75c8c349feb64451524c6536a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed44d75c8c349feb64451524c6536a6" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36361 } } peers { permanent_uuid: "6e1163382caf4551957d5c2ad39c678b" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33371 } } peers { permanent_uuid: "29153e0ab8a84b18a1db3f02cae3aefc" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41959 } }
I20260807 10:50:40.808086 14462 raft_consensus.cc:374] T 09e2bd0363354c3e9bc7d562a05e960d P 0ed44d75c8c349feb64451524c6536a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:40.808197 14462 raft_consensus.cc:729] T 09e2bd0363354c3e9bc7d562a05e960d P 0ed44d75c8c349feb64451524c6536a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ed44d75c8c349feb64451524c6536a6, State: Initialized, Role: FOLLOWER
I20260807 10:50:40.808573 14462 consensus_queue.cc:260] T 09e2bd0363354c3e9bc7d562a05e960d P 0ed44d75c8c349feb64451524c6536a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed44d75c8c349feb64451524c6536a6" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36361 } } peers { permanent_uuid: "6e1163382caf4551957d5c2ad39c678b" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33371 } } peers { permanent_uuid: "29153e0ab8a84b18a1db3f02cae3aefc" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41959 } }
I20260807 10:50:40.812978 14476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:40.800207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.3.73.193:54741)
0807 10:50:40.812281 (+ 12074us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:40.812564 (+ 283us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11880,"threads_started":1}
I20260807 10:50:40.815503 14476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:40.814230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.3.73.193:33997)
0807 10:50:40.815152 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:40.815272 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":548,"threads_started":1}
I20260807 10:50:40.816900 14478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:40.803820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.3.73.194:38715)
0807 10:50:40.816464 (+ 12644us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:40.816617 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12463,"threads_started":1}
I20260807 10:50:40.817724 14462 ts_tablet_manager.cc:1428] T 09e2bd0363354c3e9bc7d562a05e960d P 0ed44d75c8c349feb64451524c6536a6: Time spent starting tablet: real 0.011s user 0.008s sys 0.003s
I20260807 10:50:40.818668 13884 heartbeater.cc:503] Master 127.3.73.253:35079 was elected leader, sending a full tablet report...
I20260807 10:50:40.818954 14481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:40.817786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.3.73.195:55009)
0807 10:50:40.818417 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:40.818553 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":343,"threads_started":1}
I20260807 10:50:40.820129 14116 heartbeater.cc:503] Master 127.3.73.253:35079 was elected leader, sending a full tablet report...
I20260807 10:50:40.821448 14478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:40.819052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.3.73.194:42531)
0807 10:50:40.821184 (+ 2132us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:40.821281 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1997}
I20260807 10:50:40.823031 14319 heartbeater.cc:503] Master 127.3.73.253:35079 was elected leader, sending a full tablet report...
I20260807 10:50:40.823030 14481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:40.820217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.3.73.195:56941)
0807 10:50:40.822774 (+ 2557us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:40.822867 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2447}
I20260807 10:50:41.126792 14479 raft_consensus.cc:482] T 09e2bd0363354c3e9bc7d562a05e960d P 0ed44d75c8c349feb64451524c6536a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 10:50:41.127055 14479 raft_consensus.cc:504] T 09e2bd0363354c3e9bc7d562a05e960d P 0ed44d75c8c349feb64451524c6536a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed44d75c8c349feb64451524c6536a6" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36361 } } peers { permanent_uuid: "6e1163382caf4551957d5c2ad39c678b" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33371 } } peers { permanent_uuid: "29153e0ab8a84b18a1db3f02cae3aefc" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41959 } }
I20260807 10:50:41.128981 14479 leader_election.cc:288] T 09e2bd0363354c3e9bc7d562a05e960d P 0ed44d75c8c349feb64451524c6536a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e1163382caf4551957d5c2ad39c678b (127.3.73.193:33371), 29153e0ab8a84b18a1db3f02cae3aefc (127.3.73.194:41959)
I20260807 10:50:41.136507 14457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:41.129350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:41245 (local address 127.3.73.193:33371)
0807 10:50:41.129652 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:41.129659 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:41.129687 (+ 28us) server_negotiation.cc:407] Connection header received
0807 10:50:41.129822 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:41.129828 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:41.129934 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:41.130082 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:50:41.130702 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:41.131865 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:41.134363 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:41.135561 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:41.135758 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:41.135853 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:41.136160 (+ 307us) server_negotiation.cc:299] Negotiation successful
0807 10:50:41.136305 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260807 10:50:41.136622 14481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:41.129238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:33371 (local address 127.3.73.195:41245)
0807 10:50:41.129458 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:41.129483 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:41.129665 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:41.130175 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:41.130185 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:41.130225 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:50:41.130537 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:41.130551 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:41.132020 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:41.132024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:41.134116 (+ 2092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:41.134133 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:41.135775 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:41.135779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:41.135947 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:41.135973 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 10:50:41.136105 (+ 132us) client_negotiation.cc:240] Negotiation successful
0807 10:50:41.136275 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"mutex_wait_us":1}
I20260807 10:50:41.137040 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:41.129838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:51501 (local address 127.3.73.194:41959)
0807 10:50:41.130068 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:41.130075 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:41.130478 (+ 403us) server_negotiation.cc:407] Connection header received
0807 10:50:41.130713 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:41.130719 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:41.130870 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:41.130985 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:50:41.131439 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:41.132389 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:41.134749 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:41.135984 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:41.136155 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:41.136222 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:41.136606 (+ 384us) server_negotiation.cc:299] Negotiation successful
0807 10:50:41.136807 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 10:50:41.137172 14483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:41.129710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:41959 (local address 127.3.73.195:51501)
0807 10:50:41.130362 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:41.130379 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:41.130515 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:41.131002 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:41.131005 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:41.131026 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:50:41.131257 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:41.131265 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:41.132542 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:41.132549 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:41.134576 (+ 2027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:41.134587 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:41.136230 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:41.136236 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:41.136424 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:41.136456 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 10:50:41.136550 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 10:50:41.136695 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"mutex_wait_us":46,"thread_start_us":332,"threads_started":1}
I20260807 10:50:41.138051 14043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09e2bd0363354c3e9bc7d562a05e960d" candidate_uuid: "0ed44d75c8c349feb64451524c6536a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29153e0ab8a84b18a1db3f02cae3aefc" is_pre_election: true
I20260807 10:50:41.137893 13819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09e2bd0363354c3e9bc7d562a05e960d" candidate_uuid: "0ed44d75c8c349feb64451524c6536a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e1163382caf4551957d5c2ad39c678b" is_pre_election: true
I20260807 10:50:41.138330 14043 raft_consensus.cc:2454] T 09e2bd0363354c3e9bc7d562a05e960d P 29153e0ab8a84b18a1db3f02cae3aefc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ed44d75c8c349feb64451524c6536a6 in term 0.
I20260807 10:50:41.138413 13819 raft_consensus.cc:2454] T 09e2bd0363354c3e9bc7d562a05e960d P 6e1163382caf4551957d5c2ad39c678b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ed44d75c8c349feb64451524c6536a6 in term 0.
I20260807 10:50:41.138988 14158 leader_election.cc:302] T 09e2bd0363354c3e9bc7d562a05e960d P 0ed44d75c8c349feb64451524c6536a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ed44d75c8c349feb64451524c6536a6, 29153e0ab8a84b18a1db3f02cae3aefc; no voters:
I20260807 10:50:41.139402 14479 raft_consensus.cc:2781] T 09e2bd0363354c3e9bc7d562a05e960d P 0ed44d75c8c349feb64451524c6536a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 10:50:41.139514 14479 raft_consensus.cc:482] T 09e2bd0363354c3e9bc7d562a05e960d P 0ed44d75c8c349feb64451524c6536a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 10:50:41.139664 14479 raft_consensus.cc:3037] T 09e2bd0363354c3e9bc7d562a05e960d P 0ed44d75c8c349feb64451524c6536a6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:41.142041 14479 raft_consensus.cc:504] T 09e2bd0363354c3e9bc7d562a05e960d P 0ed44d75c8c349feb64451524c6536a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed44d75c8c349feb64451524c6536a6" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36361 } } peers { permanent_uuid: "6e1163382caf4551957d5c2ad39c678b" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33371 } } peers { permanent_uuid: "29153e0ab8a84b18a1db3f02cae3aefc" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41959 } }
I20260807 10:50:41.142654 14479 leader_election.cc:288] T 09e2bd0363354c3e9bc7d562a05e960d P 0ed44d75c8c349feb64451524c6536a6 [CANDIDATE]: Term 1 election: Requested vote from peers 6e1163382caf4551957d5c2ad39c678b (127.3.73.193:33371), 29153e0ab8a84b18a1db3f02cae3aefc (127.3.73.194:41959)
I20260807 10:50:41.143301 13819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09e2bd0363354c3e9bc7d562a05e960d" candidate_uuid: "0ed44d75c8c349feb64451524c6536a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e1163382caf4551957d5c2ad39c678b"
I20260807 10:50:41.143530 13819 raft_consensus.cc:3037] T 09e2bd0363354c3e9bc7d562a05e960d P 6e1163382caf4551957d5c2ad39c678b [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:41.143491 14043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09e2bd0363354c3e9bc7d562a05e960d" candidate_uuid: "0ed44d75c8c349feb64451524c6536a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29153e0ab8a84b18a1db3f02cae3aefc"
I20260807 10:50:41.143734 14043 raft_consensus.cc:3037] T 09e2bd0363354c3e9bc7d562a05e960d P 29153e0ab8a84b18a1db3f02cae3aefc [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:41.146011 14043 raft_consensus.cc:2454] T 09e2bd0363354c3e9bc7d562a05e960d P 29153e0ab8a84b18a1db3f02cae3aefc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ed44d75c8c349feb64451524c6536a6 in term 1.
I20260807 10:50:41.146025 13819 raft_consensus.cc:2454] T 09e2bd0363354c3e9bc7d562a05e960d P 6e1163382caf4551957d5c2ad39c678b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ed44d75c8c349feb64451524c6536a6 in term 1.
I20260807 10:50:41.146514 14158 leader_election.cc:302] T 09e2bd0363354c3e9bc7d562a05e960d P 0ed44d75c8c349feb64451524c6536a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ed44d75c8c349feb64451524c6536a6, 6e1163382caf4551957d5c2ad39c678b; no voters:
I20260807 10:50:41.146860 14479 raft_consensus.cc:2781] T 09e2bd0363354c3e9bc7d562a05e960d P 0ed44d75c8c349feb64451524c6536a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:50:41.147338 14479 raft_consensus.cc:686] T 09e2bd0363354c3e9bc7d562a05e960d P 0ed44d75c8c349feb64451524c6536a6 [term 1 LEADER]: Becoming Leader. State: Replica: 0ed44d75c8c349feb64451524c6536a6, State: Running, Role: LEADER
I20260807 10:50:41.147893 14479 consensus_queue.cc:237] T 09e2bd0363354c3e9bc7d562a05e960d P 0ed44d75c8c349feb64451524c6536a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed44d75c8c349feb64451524c6536a6" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36361 } } peers { permanent_uuid: "6e1163382caf4551957d5c2ad39c678b" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33371 } } peers { permanent_uuid: "29153e0ab8a84b18a1db3f02cae3aefc" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41959 } }
I20260807 10:50:41.148574 14483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:41.148019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.3.73.195:60289)
0807 10:50:41.148286 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:41.148385 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260807 10:50:41.152104 13350 catalog_manager.cc:5495] T 09e2bd0363354c3e9bc7d562a05e960d P 0ed44d75c8c349feb64451524c6536a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ed44d75c8c349feb64451524c6536a6 (127.3.73.195). New cstate: current_term: 1 leader_uuid: "0ed44d75c8c349feb64451524c6536a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ed44d75c8c349feb64451524c6536a6" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36361 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e1163382caf4551957d5c2ad39c678b" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33371 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "29153e0ab8a84b18a1db3f02cae3aefc" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41959 } health_report { overall_health: UNKNOWN } } }
I20260807 10:50:41.250651 14456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:41.245052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42210 (local address 127.3.73.195:36361)
0807 10:50:41.245262 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:41.245268 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:41.245731 (+ 463us) server_negotiation.cc:407] Connection header received
0807 10:50:41.245919 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:41.245924 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:41.246022 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:41.246159 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:50:41.246554 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:41.247280 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:41.248605 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:41.249049 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:41.249147 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:41.249516 (+ 369us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:50:41.250142 (+ 626us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:50:41.250259 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:41.250367 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 10:50:41.250438 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 10:50:41.255108 14043 raft_consensus.cc:1261] T 09e2bd0363354c3e9bc7d562a05e960d P 29153e0ab8a84b18a1db3f02cae3aefc [term 1 FOLLOWER]: Refusing update from remote peer 0ed44d75c8c349feb64451524c6536a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:50:41.255411 13819 raft_consensus.cc:1261] T 09e2bd0363354c3e9bc7d562a05e960d P 6e1163382caf4551957d5c2ad39c678b [term 1 FOLLOWER]: Refusing update from remote peer 0ed44d75c8c349feb64451524c6536a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:50:41.255945 14484 consensus_queue.cc:1035] T 09e2bd0363354c3e9bc7d562a05e960d P 0ed44d75c8c349feb64451524c6536a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29153e0ab8a84b18a1db3f02cae3aefc" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:50:41.256364 14479 consensus_queue.cc:1035] T 09e2bd0363354c3e9bc7d562a05e960d P 0ed44d75c8c349feb64451524c6536a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e1163382caf4551957d5c2ad39c678b" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33371 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:50:41.259390 14478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:41.258841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.3.73.194:43369)
0807 10:50:41.259038 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:41.259161 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260807 10:50:41.260035 14480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:41.259277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.3.73.193:35299)
0807 10:50:41.259551 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:41.259758 (+ 207us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
I20260807 10:50:41.265913 14432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:41.265368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.0.0.1:51130)
0807 10:50:41.265619 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:41.265724 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
I20260807 10:50:41.267802 14432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:41.267339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.0.0.1:51140)
0807 10:50:41.267551 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:41.267647 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260807 10:50:41.268520 14491 mvcc.cc:205] Tried to move back new op lower bound from 7315864949773418496 to 7315864949346684928. Current Snapshot: MvccSnapshot[applied={T|T < 7315864949773418496}]
I20260807 10:50:41.269290 14478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:41.268747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.3.73.194:44487)
0807 10:50:41.269071 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:41.269169 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213}
I20260807 10:50:41.270426 14483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:41.269879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.3.73.195:44647)
0807 10:50:41.270122 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:41.270237 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
I20260807 10:50:41.272934 14483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:41.272338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.3.73.195:33943)
0807 10:50:41.272636 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:41.272770 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189}
I20260807 10:50:41.276206 14480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:41.275621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.3.73.193:33431)
0807 10:50:41.275912 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:41.276030 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167}
I20260807 10:50:41.283021 13350 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:50:41.283306 13350 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:50:41.290001 13350 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 33fdc900cfa34b11aa90dc7c2cf3c84f: Sending DeleteTablet for 3 replicas of tablet 09e2bd0363354c3e9bc7d562a05e960d
I20260807 10:50:41.291080 14217 tablet_service.cc:1514] Processing DeleteTablet for tablet 09e2bd0363354c3e9bc7d562a05e960d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:50:41 UTC) from {username='oryx', principal='oryx/127.3.73.253@KRBTEST.COM'} at 127.0.0.1:42200
I20260807 10:50:41.291208 13791 tablet_service.cc:1514] Processing DeleteTablet for tablet 09e2bd0363354c3e9bc7d562a05e960d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:50:41 UTC) from {username='oryx', principal='oryx/127.3.73.253@KRBTEST.COM'} at 127.0.0.1:42504
I20260807 10:50:41.291461 14015 tablet_service.cc:1514] Processing DeleteTablet for tablet 09e2bd0363354c3e9bc7d562a05e960d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:50:41 UTC) from {username='oryx', principal='oryx/127.3.73.253@KRBTEST.COM'} at 127.0.0.1:35690
I20260807 10:50:41.297667 14506 tablet_replica.cc:330] T 09e2bd0363354c3e9bc7d562a05e960d P 6e1163382caf4551957d5c2ad39c678b: stopping tablet replica
I20260807 10:50:41.297960 14506 raft_consensus.cc:2229] T 09e2bd0363354c3e9bc7d562a05e960d P 6e1163382caf4551957d5c2ad39c678b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:50:41.298030 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 13594
I20260807 10:50:41.298368 14506 raft_consensus.cc:2258] T 09e2bd0363354c3e9bc7d562a05e960d P 6e1163382caf4551957d5c2ad39c678b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:50:41.298964 14507 tablet_replica.cc:330] T 09e2bd0363354c3e9bc7d562a05e960d P 29153e0ab8a84b18a1db3f02cae3aefc: stopping tablet replica
I20260807 10:50:41.299224 14507 raft_consensus.cc:2229] T 09e2bd0363354c3e9bc7d562a05e960d P 29153e0ab8a84b18a1db3f02cae3aefc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:50:41.299611 14507 raft_consensus.cc:2258] T 09e2bd0363354c3e9bc7d562a05e960d P 29153e0ab8a84b18a1db3f02cae3aefc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:50:41.299862 14506 ts_tablet_manager.cc:1905] T 09e2bd0363354c3e9bc7d562a05e960d P 6e1163382caf4551957d5c2ad39c678b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:50:41.301398 14507 ts_tablet_manager.cc:1905] T 09e2bd0363354c3e9bc7d562a05e960d P 29153e0ab8a84b18a1db3f02cae3aefc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:50:41.308584 14505 tablet_replica.cc:330] T 09e2bd0363354c3e9bc7d562a05e960d P 0ed44d75c8c349feb64451524c6536a6: stopping tablet replica
I20260807 10:50:41.308947 14507 ts_tablet_manager.cc:1918] T 09e2bd0363354c3e9bc7d562a05e960d P 29153e0ab8a84b18a1db3f02cae3aefc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:50:41.308940 14505 raft_consensus.cc:2229] T 09e2bd0363354c3e9bc7d562a05e960d P 0ed44d75c8c349feb64451524c6536a6 [term 1 LEADER]: Raft consensus shutting down.
I20260807 10:50:41.309222 14507 log.cc:1196] T 09e2bd0363354c3e9bc7d562a05e960d P 29153e0ab8a84b18a1db3f02cae3aefc: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-1/wal/wals/09e2bd0363354c3e9bc7d562a05e960d
I20260807 10:50:41.309648 14505 raft_consensus.cc:2258] T 09e2bd0363354c3e9bc7d562a05e960d P 0ed44d75c8c349feb64451524c6536a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:50:41.309964 14507 ts_tablet_manager.cc:1939] T 09e2bd0363354c3e9bc7d562a05e960d P 29153e0ab8a84b18a1db3f02cae3aefc: Deleting consensus metadata
I20260807 10:50:41.310696 14505 ts_tablet_manager.cc:1905] T 09e2bd0363354c3e9bc7d562a05e960d P 0ed44d75c8c349feb64451524c6536a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:50:41.311678 13332 catalog_manager.cc:4841] TS 29153e0ab8a84b18a1db3f02cae3aefc (127.3.73.194:41959): tablet 09e2bd0363354c3e9bc7d562a05e960d (table test-table [id=3120f512d26446ac90b188ac5564f904]) successfully deleted
I20260807 10:50:41.316229 14506 ts_tablet_manager.cc:1918] T 09e2bd0363354c3e9bc7d562a05e960d P 6e1163382caf4551957d5c2ad39c678b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:50:41.316478 14506 log.cc:1196] T 09e2bd0363354c3e9bc7d562a05e960d P 6e1163382caf4551957d5c2ad39c678b: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-0/wal/wals/09e2bd0363354c3e9bc7d562a05e960d
I20260807 10:50:41.317165 14506 ts_tablet_manager.cc:1939] T 09e2bd0363354c3e9bc7d562a05e960d P 6e1163382caf4551957d5c2ad39c678b: Deleting consensus metadata
I20260807 10:50:41.318135 14505 ts_tablet_manager.cc:1918] T 09e2bd0363354c3e9bc7d562a05e960d P 0ed44d75c8c349feb64451524c6536a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:50:41.318291 14505 log.cc:1196] T 09e2bd0363354c3e9bc7d562a05e960d P 0ed44d75c8c349feb64451524c6536a6: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786099767749326-3367-0/minicluster-data/ts-2/wal/wals/09e2bd0363354c3e9bc7d562a05e960d
I20260807 10:50:41.318619 13332 catalog_manager.cc:4841] TS 6e1163382caf4551957d5c2ad39c678b (127.3.73.193:33371): tablet 09e2bd0363354c3e9bc7d562a05e960d (table test-table [id=3120f512d26446ac90b188ac5564f904]) successfully deleted
I20260807 10:50:41.318792 14505 ts_tablet_manager.cc:1939] T 09e2bd0363354c3e9bc7d562a05e960d P 0ed44d75c8c349feb64451524c6536a6: Deleting consensus metadata
I20260807 10:50:41.319960 13331 catalog_manager.cc:4841] TS 0ed44d75c8c349feb64451524c6536a6 (127.3.73.195:36361): tablet 09e2bd0363354c3e9bc7d562a05e960d (table test-table [id=3120f512d26446ac90b188ac5564f904]) successfully deleted
I20260807 10:50:41.320766 14457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:41.299781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42520 (local address 127.3.73.193:33371)
0807 10:50:41.299999 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:41.300006 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:41.301518 (+ 1512us) server_negotiation.cc:407] Connection header received
0807 10:50:41.301722 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:41.301727 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:41.301820 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:41.301942 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:41.303938 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:41.305074 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:41.306956 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:41.307666 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:41.307767 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:41.312264 (+ 4497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:41.312285 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:41.312305 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:41.312364 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:41.317155 (+ 4791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:41.318141 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:41.318150 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:41.318154 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:41.318363 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:41.318863 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:41.318868 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:41.318871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:41.319295 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:41.319511 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:41.320036 (+ 525us) server_negotiation.cc:299] Negotiation successful
0807 10:50:41.320337 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 10:50:41.321640 13875 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:41.453333 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 13594
I20260807 10:50:41.495414 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 13900
I20260807 10:50:41.514770 14459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:41.496524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35692 (local address 127.3.73.194:41959)
0807 10:50:41.496752 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:41.496759 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:41.496936 (+ 177us) server_negotiation.cc:407] Connection header received
0807 10:50:41.497163 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:41.497169 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:41.497258 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:41.497362 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:41.499607 (+ 2245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:41.500810 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:41.503002 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:41.503645 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:41.503794 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:41.507339 (+ 3545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:41.507359 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:41.507379 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:41.507431 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:41.511168 (+ 3737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:41.512133 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:41.512141 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:41.512143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:41.512289 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:41.512974 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:41.512978 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:41.512980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:41.513365 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:41.513623 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:41.514082 (+ 459us) server_negotiation.cc:299] Negotiation successful
0807 10:50:41.514327 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 10:50:41.515594 14105 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:41.627929 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 13900
I20260807 10:50:41.659135 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 14130
I20260807 10:50:41.677810 14456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:41.660019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42216 (local address 127.3.73.195:36361)
0807 10:50:41.660207 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:41.660212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:41.660254 (+ 42us) server_negotiation.cc:407] Connection header received
0807 10:50:41.660522 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:41.660527 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:41.660622 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:41.660706 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:41.662852 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:41.663676 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:41.665662 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:41.666161 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:41.666287 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:41.670133 (+ 3846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:41.670152 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:41.670171 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:41.670216 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:41.674338 (+ 4122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:41.675303 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:41.675311 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:41.675314 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:41.675505 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:41.675986 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:41.675990 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:41.675992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:41.676412 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:41.676679 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:41.677189 (+ 510us) server_negotiation.cc:299] Negotiation successful
0807 10:50:41.677407 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 10:50:41.678705 14309 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:41.800465 14515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:41.799685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.0.0.1:51152)
0807 10:50:41.800129 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:41.800228 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":212,"threads_started":1}
I20260807 10:50:41.802014 14515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:41.801598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41969 (local address 127.0.0.1:51154)
0807 10:50:41.801779 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:41.801881 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:41969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 10:50:41.817322 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 14130
I20260807 10:50:41.847188 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 13136
I20260807 10:50:41.848469 13402 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:41.977425 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 13136
I20260807 10:50:42.010632 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 13299
I20260807 10:50:42.011902 13673 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:42.135531 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 13299
2026-08-07T10:50:42Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (11861 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:50:42 dist-test-slave-x5fp krb5kdc[14524](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:50:42 dist-test-slave-x5fp krb5kdc[14524](info): set up 2 sockets
Aug 07 10:50:42 dist-test-slave-x5fp krb5kdc[14524](info): commencing operation
krb5kdc: starting...
W20260807 10:50:43.527489 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.333s user 0.001s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:50:43 dist-test-slave-x5fp krb5kdc[14524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:50:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:50:43Z Disabled control of system clock
WARNING: no policy specified for oryx/127.3.73.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.73.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.73.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.254.keytab.
Entry for principal oryx/127.3.73.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.254.keytab.
I20260807 10:50:43.849164 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:42755
--webserver_interface=127.3.73.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.254.keytab
--principal=oryx/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:42611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:42755
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:50:44.224694 14541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:44.225158 14541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:44.225333 14541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:44.233009 14541 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:50:44.233160 14541 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 10:50:44.233261 14541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:44.233388 14541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:44.233479 14541 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:50:44.233577 14541 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:50:44.243873 14541 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:42611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:42755
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.254.keytab
--principal=oryx/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:42755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:44.245280 14541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:44.246661 14541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:44.259258 14548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:44.259781 14551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:44.259871 14547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:44.260510 14541 server_base.cc:1034] running on GCE node
I20260807 10:50:44.261525 14541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:44.263051 14541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:44.264343 14541 hybrid_clock.cc:648] HybridClock initialized: now 1786099844264261 us; error 46 us; skew 500 ppm
I20260807 10:50:44.264858 14541 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:44 dist-test-slave-x5fp krb5kdc[14524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099844, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:44.273319 14541 init.cc:377] Logged in from keytab as oryx/127.3.73.254@KRBTEST.COM (short username oryx)
I20260807 10:50:44.275164 14541 webserver.cc:460] Webserver started at http://127.3.73.254:46151/ using document root <none> and password file <none>
I20260807 10:50:44.276002 14541 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:44.276129 14541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:44.276521 14541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:44.280241 14541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "014a586fde6e4cd8be60dd6be039aa30"
format_stamp: "Formatted at 2026-08-07 10:50:44 on dist-test-slave-x5fp"
server_key: "276b36569c6ceda107b069bd664d3a88"
server_key_iv: "ad18638ab823b9f222bb13f244d98d8a"
server_key_version: "encryptionkey@0"
I20260807 10:50:44.281296 14541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "014a586fde6e4cd8be60dd6be039aa30"
format_stamp: "Formatted at 2026-08-07 10:50:44 on dist-test-slave-x5fp"
server_key: "276b36569c6ceda107b069bd664d3a88"
server_key_iv: "ad18638ab823b9f222bb13f244d98d8a"
server_key_version: "encryptionkey@0"
I20260807 10:50:44.287626 14541 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 10:50:44.291891 14563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:44.293677 14541 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 10:50:44.293931 14541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "014a586fde6e4cd8be60dd6be039aa30"
format_stamp: "Formatted at 2026-08-07 10:50:44 on dist-test-slave-x5fp"
server_key: "276b36569c6ceda107b069bd664d3a88"
server_key_iv: "ad18638ab823b9f222bb13f244d98d8a"
server_key_version: "encryptionkey@0"
I20260807 10:50:44.294222 14541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:44.325034 14541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:44.330036 14541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:44.330400 14541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:44.354585 14541 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:42755
I20260807 10:50:44.354604 14652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:42755 every 8 connection(s)
I20260807 10:50:44.356104 14541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:50:44.360332 14653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:44.362323 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 14541
I20260807 10:50:44.362555 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:50:44.363257 3367 external_mini_cluster.cc:1428] Setting key 0d411c7cb646c78b2d9a43974c6710a2
I20260807 10:50:44.374406 14653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 014a586fde6e4cd8be60dd6be039aa30: Bootstrap starting.
I20260807 10:50:44.378538 14653 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 014a586fde6e4cd8be60dd6be039aa30: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:44.379706 14653 log.cc:824] T 00000000000000000000000000000000 P 014a586fde6e4cd8be60dd6be039aa30: Log is configured to *not* fsync() on all Append() calls
Aug 07 10:50:44 dist-test-slave-x5fp krb5kdc[14524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM
I20260807 10:50:44.383879 14653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 014a586fde6e4cd8be60dd6be039aa30: No bootstrap required, opened a new log
I20260807 10:50:44.392067 14658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:44.365621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34446 (local address 127.3.73.254:42755)
0807 10:50:44.366544 (+ 923us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:44.366560 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:44.366635 (+ 75us) server_negotiation.cc:407] Connection header received
0807 10:50:44.369107 (+ 2472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:44.369141 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:44.369706 (+ 565us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:44.370228 (+ 522us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:44.371810 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:44.373867 (+ 2057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:44.375403 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:44.376580 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:44.376842 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:44.381794 (+ 4952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:44.381808 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:44.381830 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:44.381910 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:44.386582 (+ 4672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:44.387470 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:44.387480 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:44.387491 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:44.387735 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:44.388160 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:44.388165 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:44.388169 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:44.389036 (+ 867us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:44.389415 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:44.389839 (+ 424us) server_negotiation.cc:299] Negotiation successful
0807 10:50:44.390401 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":372,"threads_started":1}
I20260807 10:50:44.396044 14653 raft_consensus.cc:348] T 00000000000000000000000000000000 P 014a586fde6e4cd8be60dd6be039aa30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "014a586fde6e4cd8be60dd6be039aa30" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 42755 } }
I20260807 10:50:44.396471 14653 raft_consensus.cc:374] T 00000000000000000000000000000000 P 014a586fde6e4cd8be60dd6be039aa30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:44.396559 14653 raft_consensus.cc:729] T 00000000000000000000000000000000 P 014a586fde6e4cd8be60dd6be039aa30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 014a586fde6e4cd8be60dd6be039aa30, State: Initialized, Role: FOLLOWER
I20260807 10:50:44.397171 14653 consensus_queue.cc:260] T 00000000000000000000000000000000 P 014a586fde6e4cd8be60dd6be039aa30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "014a586fde6e4cd8be60dd6be039aa30" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 42755 } }
I20260807 10:50:44.397462 14653 raft_consensus.cc:388] T 00000000000000000000000000000000 P 014a586fde6e4cd8be60dd6be039aa30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:50:44.397640 14653 raft_consensus.cc:482] T 00000000000000000000000000000000 P 014a586fde6e4cd8be60dd6be039aa30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:50:44.397897 14653 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 014a586fde6e4cd8be60dd6be039aa30 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:44.400619 14653 raft_consensus.cc:504] T 00000000000000000000000000000000 P 014a586fde6e4cd8be60dd6be039aa30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "014a586fde6e4cd8be60dd6be039aa30" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 42755 } }
I20260807 10:50:44.401335 14653 leader_election.cc:302] T 00000000000000000000000000000000 P 014a586fde6e4cd8be60dd6be039aa30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 014a586fde6e4cd8be60dd6be039aa30; no voters:
I20260807 10:50:44.402011 14653 leader_election.cc:288] T 00000000000000000000000000000000 P 014a586fde6e4cd8be60dd6be039aa30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:50:44.402169 14662 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 014a586fde6e4cd8be60dd6be039aa30 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:50:44.402925 14662 raft_consensus.cc:686] T 00000000000000000000000000000000 P 014a586fde6e4cd8be60dd6be039aa30 [term 1 LEADER]: Becoming Leader. State: Replica: 014a586fde6e4cd8be60dd6be039aa30, State: Running, Role: LEADER
I20260807 10:50:44.403558 14662 consensus_queue.cc:237] T 00000000000000000000000000000000 P 014a586fde6e4cd8be60dd6be039aa30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "014a586fde6e4cd8be60dd6be039aa30" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 42755 } }
I20260807 10:50:44.404495 14653 sys_catalog.cc:564] T 00000000000000000000000000000000 P 014a586fde6e4cd8be60dd6be039aa30 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:50:44.406569 14664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 014a586fde6e4cd8be60dd6be039aa30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 014a586fde6e4cd8be60dd6be039aa30. Latest consensus state: current_term: 1 leader_uuid: "014a586fde6e4cd8be60dd6be039aa30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "014a586fde6e4cd8be60dd6be039aa30" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 42755 } } }
I20260807 10:50:44.406509 14663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 014a586fde6e4cd8be60dd6be039aa30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "014a586fde6e4cd8be60dd6be039aa30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "014a586fde6e4cd8be60dd6be039aa30" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 42755 } } }
I20260807 10:50:44.406977 14663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 014a586fde6e4cd8be60dd6be039aa30 [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:44.406977 14664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 014a586fde6e4cd8be60dd6be039aa30 [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:44.408924 14669 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:50:44.414487 14669 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:50:44.423908 14669 catalog_manager.cc:1347] Generated new cluster ID: 625f91fea5184eb08dc4be0796869355
I20260807 10:50:44.424053 14669 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:50:44.450187 14669 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:50:44.451594 14669 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:50:44.462832 14669 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 014a586fde6e4cd8be60dd6be039aa30: Generated new TSK 0
I20260807 10:50:44.463924 14669 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.3.73.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.73.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.73.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.193.keytab.
Entry for principal oryx/127.3.73.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.193.keytab.
I20260807 10:50:44.542640 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.193:0
--local_ip_for_outbound_sockets=127.3.73.193
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:42755
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.193.keytab
--principal=oryx/127.3.73.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:42611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:50:44.910876 14692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:44.911269 14692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:44.911448 14692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:44.919421 14692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:44.919559 14692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:44.919710 14692 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 10:50:44.919853 14692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.193
I20260807 10:50:44.930320 14692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:42611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.193.keytab
--principal=oryx/127.3.73.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.73.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.73.254:42755
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.193
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:44.931779 14692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:44.933305 14692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:44.947489 14700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:44.947449 14707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:44.947554 14702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:44.949265 14692 server_base.cc:1034] running on GCE node
I20260807 10:50:44.950022 14692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:44.952111 14692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:44.953408 14692 hybrid_clock.cc:648] HybridClock initialized: now 1786099844953312 us; error 69 us; skew 500 ppm
I20260807 10:50:44.953794 14692 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:44 dist-test-slave-x5fp krb5kdc[14524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099844, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:44.957473 14692 init.cc:377] Logged in from keytab as oryx/127.3.73.193@KRBTEST.COM (short username oryx)
I20260807 10:50:44.959084 14692 webserver.cc:460] Webserver started at http://127.3.73.193:34171/ using document root <none> and password file <none>
I20260807 10:50:44.959853 14692 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:44.959978 14692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:44.960340 14692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:44.963503 14692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-0/data/instance:
uuid: "2fcbcf1b2e12435999b48339cbfbd77c"
format_stamp: "Formatted at 2026-08-07 10:50:44 on dist-test-slave-x5fp"
server_key: "0f298f9a17dbf8835573cf9a7a0ed6a6"
server_key_iv: "8a46013d5190dc0584f3c5457fc753d1"
server_key_version: "encryptionkey@0"
I20260807 10:50:44.964336 14692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance:
uuid: "2fcbcf1b2e12435999b48339cbfbd77c"
format_stamp: "Formatted at 2026-08-07 10:50:44 on dist-test-slave-x5fp"
server_key: "0f298f9a17dbf8835573cf9a7a0ed6a6"
server_key_iv: "8a46013d5190dc0584f3c5457fc753d1"
server_key_version: "encryptionkey@0"
I20260807 10:50:44.970089 14692 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260807 10:50:44.974025 14715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:44.975625 14692 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 10:50:44.975903 14692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-0/wal
uuid: "2fcbcf1b2e12435999b48339cbfbd77c"
format_stamp: "Formatted at 2026-08-07 10:50:44 on dist-test-slave-x5fp"
server_key: "0f298f9a17dbf8835573cf9a7a0ed6a6"
server_key_iv: "8a46013d5190dc0584f3c5457fc753d1"
server_key_version: "encryptionkey@0"
I20260807 10:50:44.976192 14692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:45.003309 14692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:45.008472 14692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:45.008869 14692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:45.012272 14692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:50:45.012400 14692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:45.012598 14692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:50:45.012687 14692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 10:50:45 dist-test-slave-x5fp krb5kdc[14524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099844, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM
I20260807 10:50:45.039849 14727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:45.012960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:50869 (local address 127.3.73.254:42755)
0807 10:50:45.014836 (+ 1876us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:45.014841 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:45.015274 (+ 433us) server_negotiation.cc:407] Connection header received
0807 10:50:45.017740 (+ 2466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:45.017750 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:45.017872 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:45.018003 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:45.020082 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:45.021436 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:45.023556 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:45.024465 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:45.024638 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:45.030511 (+ 5873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:45.030517 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:45.030521 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:45.030558 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:45.034248 (+ 3690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:45.035608 (+ 1360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:45.035617 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:45.035624 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:45.035932 (+ 308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:45.036626 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:45.036632 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:45.036636 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:45.037109 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:45.037277 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:45.039018 (+ 1741us) server_negotiation.cc:299] Negotiation successful
0807 10:50:45.039322 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1753,"thread_start_us":1633,"threads_started":1}
I20260807 10:50:45.040989 14730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:45.013937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:42755 (local address 127.3.73.193:50869)
0807 10:50:45.015066 (+ 1129us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:45.015113 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:45.017397 (+ 2284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:45.018316 (+ 919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:45.018327 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:45.019227 (+ 900us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:45.019860 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:45.019879 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:45.021657 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:45.021665 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:45.023380 (+ 1715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:45.023395 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:45.024824 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:45.024835 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:45.025206 (+ 371us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:45.026178 (+ 972us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:45.026222 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:45.030147 (+ 3925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:45.034455 (+ 4308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:45.034470 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:45.034509 (+ 39us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:45.035446 (+ 937us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:45.036101 (+ 655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:45.036110 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:45.036114 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:45.036500 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:45.037353 (+ 853us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:45.037363 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:45.037967 (+ 604us) client_negotiation.cc:769] Sending connection context
0807 10:50:45.038381 (+ 414us) client_negotiation.cc:240] Negotiation successful
0807 10:50:45.039068 (+ 687us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":796,"thread_start_us":516,"threads_started":1}
I20260807 10:50:45.061254 14692 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.193:43623
I20260807 10:50:45.061283 14896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.193:43623 every 8 connection(s)
I20260807 10:50:45.063035 14692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
I20260807 10:50:45.065704 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 14692
I20260807 10:50:45.065953 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance
I20260807 10:50:45.066596 14727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:45.048998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:33141 (local address 127.3.73.254:42755)
0807 10:50:45.049250 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:45.049259 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:45.050153 (+ 894us) server_negotiation.cc:407] Connection header received
0807 10:50:45.050326 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:45.050333 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:45.050429 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:45.050554 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:45.052304 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:45.053416 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:45.055004 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:45.055593 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:45.055735 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:45.058858 (+ 3123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:45.058866 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:45.058870 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:45.058913 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:45.062695 (+ 3782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:45.063565 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:45.063574 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:45.063580 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:45.063765 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:45.064218 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:45.064226 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:45.064232 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:45.064638 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:45.064829 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:45.066184 (+ 1355us) server_negotiation.cc:299] Negotiation successful
0807 10:50:45.066366 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 10:50:45.066639 3367 external_mini_cluster.cc:1428] Setting key 2503a5b03df1d2a97f59e5b05024fc8c
I20260807 10:50:45.065979 14860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:45.049088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:42755 (local address 127.3.73.193:33141)
0807 10:50:45.050016 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:45.050046 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:45.050196 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:45.050612 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:45.050616 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:45.051496 (+ 880us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:45.052080 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:45.052094 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:45.053560 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:45.053566 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:45.054827 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:45.054841 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:45.055816 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:45.055824 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:45.055996 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:45.056954 (+ 958us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:45.056992 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:45.058552 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:45.062855 (+ 4303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:45.062859 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:45.062862 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:45.063416 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:45.063904 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:45.063909 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:45.063912 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:45.064074 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:45.064783 (+ 709us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:45.064788 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:45.065004 (+ 216us) client_negotiation.cc:769] Sending connection context
0807 10:50:45.065195 (+ 191us) client_negotiation.cc:240] Negotiation successful
0807 10:50:45.065512 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":763,"thread_start_us":602,"threads_started":1}
I20260807 10:50:45.067267 14897 heartbeater.cc:347] Connected to a master server at 127.3.73.254:42755
I20260807 10:50:45.068109 14897 heartbeater.cc:464] Registering TS with master...
I20260807 10:50:45.069072 14897 heartbeater.cc:511] Master 127.3.73.254:42755 requested a full tablet report, sending...
I20260807 10:50:45.071403 14585 ts_manager.cc:194] Registered new tserver with Master: 2fcbcf1b2e12435999b48339cbfbd77c (127.3.73.193:43623)
I20260807 10:50:45.073050 14585 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.73.193@KRBTEST.COM'} at 127.3.73.193:50869
WARNING: no policy specified for oryx/127.3.73.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.73.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.73.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.194.keytab.
Entry for principal oryx/127.3.73.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.194.keytab.
I20260807 10:50:45.136639 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.194:0
--local_ip_for_outbound_sockets=127.3.73.194
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:42755
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.194.keytab
--principal=oryx/127.3.73.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:42611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:50:45.526235 14903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:45.526566 14903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:45.526674 14903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:45.534584 14903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:45.534722 14903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:45.534811 14903 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 10:50:45.534919 14903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.194
I20260807 10:50:45.545360 14903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:42611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.194.keytab
--principal=oryx/127.3.73.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.73.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.73.254:42755
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.194
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:45.546926 14903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:45.548498 14903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:45.562875 14916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:45.562969 14910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:45.562922 14911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:45.563863 14903 server_base.cc:1034] running on GCE node
I20260807 10:50:45.564638 14903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:45.565734 14903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:45.567056 14903 hybrid_clock.cc:648] HybridClock initialized: now 1786099845566969 us; error 61 us; skew 500 ppm
I20260807 10:50:45.567467 14903 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:45 dist-test-slave-x5fp krb5kdc[14524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099845, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:45.571039 14903 init.cc:377] Logged in from keytab as oryx/127.3.73.194@KRBTEST.COM (short username oryx)
I20260807 10:50:45.572842 14903 webserver.cc:460] Webserver started at http://127.3.73.194:43785/ using document root <none> and password file <none>
I20260807 10:50:45.573597 14903 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:45.573694 14903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:45.573992 14903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:45.577394 14903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-1/data/instance:
uuid: "4ef46d7a4eec4247881e5ae633b95839"
format_stamp: "Formatted at 2026-08-07 10:50:45 on dist-test-slave-x5fp"
server_key: "32e66d5015aeaba1e8008681ec9635fd"
server_key_iv: "2f1e179d3be4bac5e487c3fea2abeec3"
server_key_version: "encryptionkey@0"
I20260807 10:50:45.578294 14903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance:
uuid: "4ef46d7a4eec4247881e5ae633b95839"
format_stamp: "Formatted at 2026-08-07 10:50:45 on dist-test-slave-x5fp"
server_key: "32e66d5015aeaba1e8008681ec9635fd"
server_key_iv: "2f1e179d3be4bac5e487c3fea2abeec3"
server_key_version: "encryptionkey@0"
I20260807 10:50:45.584288 14903 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 10:50:45.588140 14925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:45.589839 14903 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 10:50:45.590031 14903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-1/wal
uuid: "4ef46d7a4eec4247881e5ae633b95839"
format_stamp: "Formatted at 2026-08-07 10:50:45 on dist-test-slave-x5fp"
server_key: "32e66d5015aeaba1e8008681ec9635fd"
server_key_iv: "2f1e179d3be4bac5e487c3fea2abeec3"
server_key_version: "encryptionkey@0"
I20260807 10:50:45.590278 14903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:45.615573 14903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:45.620544 14903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:45.620913 14903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:45.624099 14903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:50:45.624231 14903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:45.624467 14903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:50:45.624594 14903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 10:50:45 dist-test-slave-x5fp krb5kdc[14524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099845, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM
I20260807 10:50:45.655193 14938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:45.624541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:43253 (local address 127.3.73.254:42755)
0807 10:50:45.625174 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:45.625183 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:45.626815 (+ 1632us) server_negotiation.cc:407] Connection header received
0807 10:50:45.629241 (+ 2426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:45.629248 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:45.629351 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:45.629503 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:45.632294 (+ 2791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:45.633704 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:45.637521 (+ 3817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:45.638428 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:45.638600 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:45.645907 (+ 7307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:45.645917 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:45.645926 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:45.645987 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:45.650399 (+ 4412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:45.651545 (+ 1146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:45.651555 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:45.651562 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:45.651886 (+ 324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:45.652587 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:45.652596 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:45.652602 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:45.653113 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:45.653321 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:45.654456 (+ 1135us) server_negotiation.cc:299] Negotiation successful
0807 10:50:45.654731 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":304,"threads_started":1}
I20260807 10:50:45.657032 14940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:45.625256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:42755 (local address 127.3.73.194:43253)
0807 10:50:45.626601 (+ 1345us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:45.626662 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:45.628798 (+ 2136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:45.629902 (+ 1104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:45.629914 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:45.630953 (+ 1039us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:45.632039 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:45.632082 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:45.633969 (+ 1887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:45.633978 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:45.637222 (+ 3244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:45.637250 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:45.639042 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:45.639054 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:45.639507 (+ 453us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:45.640666 (+ 1159us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:45.640770 (+ 104us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:45.645551 (+ 4781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:45.650659 (+ 5108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:45.650670 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:45.650701 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:45.651345 (+ 644us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:45.652064 (+ 719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:45.652069 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:45.652076 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:45.652369 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:45.653315 (+ 946us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:45.653324 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:45.653862 (+ 538us) client_negotiation.cc:769] Sending connection context
0807 10:50:45.654345 (+ 483us) client_negotiation.cc:240] Negotiation successful
0807 10:50:45.655041 (+ 696us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1028,"thread_start_us":482,"threads_started":1}
I20260807 10:50:45.670467 14903 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.194:46501
I20260807 10:50:45.670518 15104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.194:46501 every 8 connection(s)
I20260807 10:50:45.671846 14903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
I20260807 10:50:45.675269 15105 heartbeater.cc:347] Connected to a master server at 127.3.73.254:42755
I20260807 10:50:45.675617 15105 heartbeater.cc:464] Registering TS with master...
I20260807 10:50:45.676532 15105 heartbeater.cc:511] Master 127.3.73.254:42755 requested a full tablet report, sending...
I20260807 10:50:45.678462 14585 ts_manager.cc:194] Registered new tserver with Master: 4ef46d7a4eec4247881e5ae633b95839 (127.3.73.194:46501)
I20260807 10:50:45.679546 14585 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.73.194@KRBTEST.COM'} at 127.3.73.194:43253
I20260807 10:50:45.679587 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 14903
I20260807 10:50:45.679821 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance
I20260807 10:50:45.680471 3367 external_mini_cluster.cc:1428] Setting key 18cc477a3f84818bc22aacabc6bc1fd7
I20260807 10:50:45.681727 14938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:45.666451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:51285 (local address 127.3.73.254:42755)
0807 10:50:45.666696 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:45.666703 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:45.666727 (+ 24us) server_negotiation.cc:407] Connection header received
0807 10:50:45.666952 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:45.666958 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:45.667043 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:45.667163 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:45.668562 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:45.669632 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:45.671321 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:45.671785 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:45.671941 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:45.674429 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:45.674435 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:45.674440 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:45.674483 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:45.678251 (+ 3768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:45.679132 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:45.679141 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:45.679147 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:45.679332 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:45.679833 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:45.679838 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:45.679841 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:45.680273 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:45.680505 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:45.681089 (+ 584us) server_negotiation.cc:299] Negotiation successful
0807 10:50:45.681348 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 10:50:45.681713 15092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:45.665545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:42755 (local address 127.3.73.194:51285)
0807 10:50:45.666576 (+ 1031us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:45.666601 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:45.666794 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:45.667204 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:45.667209 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:45.668009 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:45.668392 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:45.668404 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:45.669792 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:45.669802 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:45.671159 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:45.671174 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:45.672000 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:45.672008 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:45.672200 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:45.673073 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:45.673093 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:45.674150 (+ 1057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:45.678399 (+ 4249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:45.678407 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:45.678413 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:45.678960 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:45.679485 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:45.679491 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:45.679495 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:45.679682 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:45.680405 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:45.680410 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:45.680653 (+ 243us) client_negotiation.cc:769] Sending connection context
0807 10:50:45.680867 (+ 214us) client_negotiation.cc:240] Negotiation successful
0807 10:50:45.681230 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":857,"thread_start_us":951,"threads_started":1}
WARNING: no policy specified for oryx/127.3.73.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.73.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.73.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.195.keytab.
Entry for principal oryx/127.3.73.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.195.keytab.
I20260807 10:50:45.753146 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.195:0
--local_ip_for_outbound_sockets=127.3.73.195
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:42755
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.195.keytab
--principal=oryx/127.3.73.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:42611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 10:50:46.076514 14897 heartbeater.cc:503] Master 127.3.73.254:42755 was elected leader, sending a full tablet report...
W20260807 10:50:46.119185 15111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:46.119520 15111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:46.119701 15111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:46.127214 15111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:46.127339 15111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:46.127491 15111 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 10:50:46.127630 15111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.195
I20260807 10:50:46.137822 15111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:42611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.195.keytab
--principal=oryx/127.3.73.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.73.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.73.254:42755
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.195
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:46.139209 15111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:46.140687 15111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:46.154708 15118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:46.154745 15119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:46.154824 15121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:46.155789 15111 server_base.cc:1034] running on GCE node
I20260807 10:50:46.156591 15111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:46.157871 15111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:46.159157 15111 hybrid_clock.cc:648] HybridClock initialized: now 1786099846159067 us; error 69 us; skew 500 ppm
I20260807 10:50:46.159476 15111 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:46 dist-test-slave-x5fp krb5kdc[14524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099846, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:46.163204 15111 init.cc:377] Logged in from keytab as oryx/127.3.73.195@KRBTEST.COM (short username oryx)
I20260807 10:50:46.165019 15111 webserver.cc:460] Webserver started at http://127.3.73.195:43391/ using document root <none> and password file <none>
I20260807 10:50:46.165789 15111 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:46.165910 15111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:46.166268 15111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:46.169616 15111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-2/data/instance:
uuid: "cad93d47780840d69dae51ac99bd7bff"
format_stamp: "Formatted at 2026-08-07 10:50:46 on dist-test-slave-x5fp"
server_key: "8d65977a3fd0c4002a9aa62876e2086b"
server_key_iv: "afdbdb2261489fcb741199bce318f76d"
server_key_version: "encryptionkey@0"
I20260807 10:50:46.170485 15111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance:
uuid: "cad93d47780840d69dae51ac99bd7bff"
format_stamp: "Formatted at 2026-08-07 10:50:46 on dist-test-slave-x5fp"
server_key: "8d65977a3fd0c4002a9aa62876e2086b"
server_key_iv: "afdbdb2261489fcb741199bce318f76d"
server_key_version: "encryptionkey@0"
I20260807 10:50:46.176252 15111 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260807 10:50:46.180497 15128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:46.182370 15111 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 10:50:46.182562 15111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-2/wal
uuid: "cad93d47780840d69dae51ac99bd7bff"
format_stamp: "Formatted at 2026-08-07 10:50:46 on dist-test-slave-x5fp"
server_key: "8d65977a3fd0c4002a9aa62876e2086b"
server_key_iv: "afdbdb2261489fcb741199bce318f76d"
server_key_version: "encryptionkey@0"
I20260807 10:50:46.182824 15111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:46.213558 15111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:46.218375 15111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:46.218693 15111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:46.221756 15111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:50:46.221841 15111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:46.222046 15111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:50:46.222110 15111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 10:50:46 dist-test-slave-x5fp krb5kdc[14524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099846, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM
I20260807 10:50:46.249416 15143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:46.222500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:60567 (local address 127.3.73.254:42755)
0807 10:50:46.223182 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:46.223190 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:46.224427 (+ 1237us) server_negotiation.cc:407] Connection header received
0807 10:50:46.227681 (+ 3254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:46.227687 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:46.227838 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:46.227984 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:46.230300 (+ 2316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.231192 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.233883 (+ 2691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.234442 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.234585 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:46.240352 (+ 5767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:46.240362 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:46.240368 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:46.240419 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:46.245007 (+ 4588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:46.246234 (+ 1227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:46.246244 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:46.246250 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:46.246462 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:46.246997 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:46.247005 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:46.247011 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:46.247523 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:46.247712 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:46.248501 (+ 789us) server_negotiation.cc:299] Negotiation successful
0807 10:50:46.248848 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":408,"threads_started":1}
I20260807 10:50:46.250363 15145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:46.223200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:42755 (local address 127.3.73.195:60567)
0807 10:50:46.224205 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:46.224273 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:46.227172 (+ 2899us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:46.228270 (+ 1098us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:46.228278 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:46.229210 (+ 932us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:46.230068 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:46.230102 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.231362 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.231368 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:46.233721 (+ 2353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:46.233736 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.234802 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.234809 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:46.235224 (+ 415us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:46.236189 (+ 965us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:46.236220 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:46.239974 (+ 3754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:46.245281 (+ 5307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:46.245294 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:46.245320 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:46.246096 (+ 776us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:46.246604 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:46.246614 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:46.246618 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:46.246853 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:46.247685 (+ 832us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:46.247693 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:46.248081 (+ 388us) client_negotiation.cc:769] Sending connection context
0807 10:50:46.248324 (+ 243us) client_negotiation.cc:240] Negotiation successful
0807 10:50:46.248808 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"thread_start_us":376,"threads_started":1}
I20260807 10:50:46.275326 15143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:46.258636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:37025 (local address 127.3.73.254:42755)
0807 10:50:46.258899 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:46.258908 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:46.259286 (+ 378us) server_negotiation.cc:407] Connection header received
0807 10:50:46.259609 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:46.259617 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:46.259712 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:46.259775 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:46.261491 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.262900 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.265053 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.265758 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.265915 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:46.267911 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:46.267917 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:46.267924 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:46.267974 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:46.272182 (+ 4208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:46.273263 (+ 1081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:46.273270 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:46.273276 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:46.273415 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:46.273805 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:46.273810 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:46.273813 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:46.274093 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:46.274288 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:46.274706 (+ 418us) server_negotiation.cc:299] Negotiation successful
0807 10:50:46.274936 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 10:50:46.275400 15253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:46.258112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:42755 (local address 127.3.73.195:37025)
0807 10:50:46.259145 (+ 1033us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:46.259173 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:46.259391 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:46.260043 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:46.260049 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:46.260830 (+ 781us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:46.261298 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:46.261318 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.263117 (+ 1799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.263124 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:46.264899 (+ 1775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:46.264914 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.266031 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.266043 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:46.266206 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:46.266925 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:46.266943 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:46.267709 (+ 766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:46.272345 (+ 4636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:46.272350 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:46.272354 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:46.273124 (+ 770us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:46.273532 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:46.273537 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:46.273541 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:46.273675 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:46.274219 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:46.274224 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:46.274423 (+ 199us) client_negotiation.cc:769] Sending connection context
0807 10:50:46.274578 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 10:50:46.274936 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":875,"thread_start_us":698,"threads_started":1}
I20260807 10:50:46.280807 15111 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.195:41029
I20260807 10:50:46.280846 15311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.195:41029 every 8 connection(s)
I20260807 10:50:46.282088 15111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
I20260807 10:50:46.284482 15313 heartbeater.cc:347] Connected to a master server at 127.3.73.254:42755
I20260807 10:50:46.284826 15313 heartbeater.cc:464] Registering TS with master...
I20260807 10:50:46.285531 15313 heartbeater.cc:511] Master 127.3.73.254:42755 requested a full tablet report, sending...
I20260807 10:50:46.287343 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 15111
I20260807 10:50:46.287349 14585 ts_manager.cc:194] Registered new tserver with Master: cad93d47780840d69dae51ac99bd7bff (127.3.73.195:41029)
I20260807 10:50:46.287570 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance
I20260807 10:50:46.288118 3367 external_mini_cluster.cc:1428] Setting key a74fbd5015faee2a00b08c025cc82241
I20260807 10:50:46.288442 14585 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.73.195@KRBTEST.COM'} at 127.3.73.195:60567
I20260807 10:50:46.291531 3367 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 10:50:46 dist-test-slave-x5fp krb5kdc[14524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099846, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 10:50:46 dist-test-slave-x5fp krb5kdc[14524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:50:46 dist-test-slave-x5fp krb5kdc[14524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 10:50:46.368163 15143 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:50:46.354125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34448 (local address 127.3.73.254:42755)
0807 10:50:46.354345 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:46.354353 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:46.354638 (+ 285us) server_negotiation.cc:407] Connection header received
0807 10:50:46.354852 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:46.354858 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:46.354969 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:46.355118 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:46.357123 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.358241 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.359977 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.360738 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.360869 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:46.367629 (+ 6760us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34448: BlockingRecv error: recv got EOF from 127.0.0.1:34448 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 07 10:50:46 dist-test-slave-x5fp krb5kdc[14524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099846, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM
I20260807 10:50:46.389401 15143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:46.372113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34460 (local address 127.3.73.254:42755)
0807 10:50:46.372378 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:46.372389 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:46.372969 (+ 580us) server_negotiation.cc:407] Connection header received
0807 10:50:46.373141 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:46.373145 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:46.373214 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:46.373314 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:46.374738 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.375487 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.376992 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.377468 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.377583 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:46.382389 (+ 4806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:46.382396 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:46.382399 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:46.382443 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:46.385441 (+ 2998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:46.386365 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:46.386371 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:46.386373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:46.386560 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:46.387212 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:46.387218 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:46.387221 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:46.387713 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:46.387982 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:46.388592 (+ 610us) server_negotiation.cc:299] Negotiation successful
0807 10:50:46.388970 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 10:50:46.394568 14585 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34460:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 10:50:46.402428 14585 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 10:50:46.443441 15346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:46.418691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39488 (local address 127.3.73.195:41029)
0807 10:50:46.432086 (+ 13395us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:46.432097 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:46.433150 (+ 1053us) server_negotiation.cc:407] Connection header received
0807 10:50:46.433527 (+ 377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:46.433539 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:46.433815 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:46.434032 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:50:46.435139 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.437179 (+ 2040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.439820 (+ 2641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.441961 (+ 2141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.442098 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:46.442343 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:46.442922 (+ 579us) server_negotiation.cc:299] Negotiation successful
0807 10:50:46.443102 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13200,"thread_start_us":13140,"threads_started":1}
I20260807 10:50:46.443423 15345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:46.418155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37446 (local address 127.3.73.193:43623)
0807 10:50:46.431430 (+ 13275us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:46.431442 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:46.431481 (+ 39us) server_negotiation.cc:407] Connection header received
0807 10:50:46.432263 (+ 782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:46.432271 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:46.432647 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:46.432914 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:50:46.434210 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.436166 (+ 1956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.440156 (+ 3990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.441653 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.441793 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:46.442024 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:46.442807 (+ 783us) server_negotiation.cc:299] Negotiation successful
0807 10:50:46.443031 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13055,"thread_start_us":14966,"threads_started":1}
I20260807 10:50:46.443462 15343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:46.417793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:43623 (local address 127.0.0.1:37446)
0807 10:50:46.431253 (+ 13460us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:46.431328 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:46.431621 (+ 293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:46.433428 (+ 1807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:46.433440 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:46.433478 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:50:46.433988 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:46.434005 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.436312 (+ 2307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.436319 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:46.439930 (+ 3611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:46.439947 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.442200 (+ 2253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.442207 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:46.442534 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:46.442583 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 10:50:46.442737 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 10:50:46.442986 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13025,"mutex_wait_us":108,"thread_start_us":12726,"threads_started":1}
I20260807 10:50:46.443646 15352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:46.432395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57538 (local address 127.3.73.194:46501)
0807 10:50:46.434674 (+ 2279us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:46.434680 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:46.434704 (+ 24us) server_negotiation.cc:407] Connection header received
0807 10:50:46.434786 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:46.434790 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:46.434950 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:46.435189 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:50:46.436069 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.437459 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.440295 (+ 2836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.441551 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.441725 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:46.441994 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:46.443145 (+ 1151us) server_negotiation.cc:299] Negotiation successful
0807 10:50:46.443340 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2122,"thread_start_us":1973,"threads_started":1}
I20260807 10:50:46.444016 15344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:46.432101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:46501 (local address 127.0.0.1:57538)
0807 10:50:46.433057 (+ 956us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:46.433083 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:46.433255 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:46.435193 (+ 1938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:46.435198 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:46.435226 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:50:46.435742 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:46.435758 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.437617 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.437624 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:46.440132 (+ 2508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:46.440142 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.442179 (+ 2037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.442184 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:46.442552 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:46.442613 (+ 61us) client_negotiation.cc:769] Sending connection context
0807 10:50:46.442720 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 10:50:46.442980 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":753,"mutex_wait_us":119,"thread_start_us":1249,"threads_started":1}
I20260807 10:50:46.444633 15351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:46.418010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:41029 (local address 127.0.0.1:39488)
0807 10:50:46.432984 (+ 14974us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:46.433021 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:46.433285 (+ 264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:46.434428 (+ 1143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:46.434436 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:46.434467 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:50:46.434886 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:46.434901 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.437432 (+ 2531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.437438 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:46.439614 (+ 2176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:46.439630 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.442467 (+ 2837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.442475 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:46.442676 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:46.442720 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 10:50:46.442806 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 10:50:46.444137 (+ 1331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14731,"mutex_wait_us":1188,"spinlock_wait_cycles":3840,"thread_start_us":15599,"threads_started":1}
I20260807 10:50:46.448400 14793 tablet_service.cc:1467] Processing CreateTablet for tablet c9ff5db66b884543b39d73cd3d905c66 (DEFAULT_TABLE table=test-table [id=d7f9f101d957493abdcd75df8a506c0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:50:46.448412 15200 tablet_service.cc:1467] Processing CreateTablet for tablet c9ff5db66b884543b39d73cd3d905c66 (DEFAULT_TABLE table=test-table [id=d7f9f101d957493abdcd75df8a506c0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:50:46.448412 14997 tablet_service.cc:1467] Processing CreateTablet for tablet c9ff5db66b884543b39d73cd3d905c66 (DEFAULT_TABLE table=test-table [id=d7f9f101d957493abdcd75df8a506c0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:50:46.449887 14793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9ff5db66b884543b39d73cd3d905c66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:46.449941 15200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9ff5db66b884543b39d73cd3d905c66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:46.449941 14997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9ff5db66b884543b39d73cd3d905c66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:46.463325 15353 tablet_bootstrap.cc:492] T c9ff5db66b884543b39d73cd3d905c66 P 2fcbcf1b2e12435999b48339cbfbd77c: Bootstrap starting.
I20260807 10:50:46.470362 15355 tablet_bootstrap.cc:492] T c9ff5db66b884543b39d73cd3d905c66 P 4ef46d7a4eec4247881e5ae633b95839: Bootstrap starting.
I20260807 10:50:46.471272 15353 tablet_bootstrap.cc:654] T c9ff5db66b884543b39d73cd3d905c66 P 2fcbcf1b2e12435999b48339cbfbd77c: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:46.474030 15353 log.cc:824] T c9ff5db66b884543b39d73cd3d905c66 P 2fcbcf1b2e12435999b48339cbfbd77c: Log is configured to *not* fsync() on all Append() calls
I20260807 10:50:46.481235 15354 tablet_bootstrap.cc:492] T c9ff5db66b884543b39d73cd3d905c66 P cad93d47780840d69dae51ac99bd7bff: Bootstrap starting.
I20260807 10:50:46.488375 15355 tablet_bootstrap.cc:654] T c9ff5db66b884543b39d73cd3d905c66 P 4ef46d7a4eec4247881e5ae633b95839: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:46.489717 15355 log.cc:824] T c9ff5db66b884543b39d73cd3d905c66 P 4ef46d7a4eec4247881e5ae633b95839: Log is configured to *not* fsync() on all Append() calls
I20260807 10:50:46.490855 15354 tablet_bootstrap.cc:654] T c9ff5db66b884543b39d73cd3d905c66 P cad93d47780840d69dae51ac99bd7bff: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:46.492798 15354 log.cc:824] T c9ff5db66b884543b39d73cd3d905c66 P cad93d47780840d69dae51ac99bd7bff: Log is configured to *not* fsync() on all Append() calls
I20260807 10:50:46.497537 15353 tablet_bootstrap.cc:492] T c9ff5db66b884543b39d73cd3d905c66 P 2fcbcf1b2e12435999b48339cbfbd77c: No bootstrap required, opened a new log
I20260807 10:50:46.497537 15355 tablet_bootstrap.cc:492] T c9ff5db66b884543b39d73cd3d905c66 P 4ef46d7a4eec4247881e5ae633b95839: No bootstrap required, opened a new log
I20260807 10:50:46.497903 15355 ts_tablet_manager.cc:1397] T c9ff5db66b884543b39d73cd3d905c66 P 4ef46d7a4eec4247881e5ae633b95839: Time spent bootstrapping tablet: real 0.028s user 0.008s sys 0.004s
I20260807 10:50:46.498019 15353 ts_tablet_manager.cc:1397] T c9ff5db66b884543b39d73cd3d905c66 P 2fcbcf1b2e12435999b48339cbfbd77c: Time spent bootstrapping tablet: real 0.035s user 0.008s sys 0.017s
I20260807 10:50:46.499012 15354 tablet_bootstrap.cc:492] T c9ff5db66b884543b39d73cd3d905c66 P cad93d47780840d69dae51ac99bd7bff: No bootstrap required, opened a new log
I20260807 10:50:46.499513 15354 ts_tablet_manager.cc:1397] T c9ff5db66b884543b39d73cd3d905c66 P cad93d47780840d69dae51ac99bd7bff: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.005s
I20260807 10:50:46.514652 15354 raft_consensus.cc:348] T c9ff5db66b884543b39d73cd3d905c66 P cad93d47780840d69dae51ac99bd7bff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } }
I20260807 10:50:46.515412 15354 raft_consensus.cc:374] T c9ff5db66b884543b39d73cd3d905c66 P cad93d47780840d69dae51ac99bd7bff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:46.515712 15354 raft_consensus.cc:729] T c9ff5db66b884543b39d73cd3d905c66 P cad93d47780840d69dae51ac99bd7bff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cad93d47780840d69dae51ac99bd7bff, State: Initialized, Role: FOLLOWER
I20260807 10:50:46.516216 15353 raft_consensus.cc:348] T c9ff5db66b884543b39d73cd3d905c66 P 2fcbcf1b2e12435999b48339cbfbd77c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } }
I20260807 10:50:46.516656 15354 consensus_queue.cc:260] T c9ff5db66b884543b39d73cd3d905c66 P cad93d47780840d69dae51ac99bd7bff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } }
I20260807 10:50:46.516948 15353 raft_consensus.cc:374] T c9ff5db66b884543b39d73cd3d905c66 P 2fcbcf1b2e12435999b48339cbfbd77c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:46.517189 15353 raft_consensus.cc:729] T c9ff5db66b884543b39d73cd3d905c66 P 2fcbcf1b2e12435999b48339cbfbd77c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fcbcf1b2e12435999b48339cbfbd77c, State: Initialized, Role: FOLLOWER
I20260807 10:50:46.516980 15355 raft_consensus.cc:348] T c9ff5db66b884543b39d73cd3d905c66 P 4ef46d7a4eec4247881e5ae633b95839 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } }
I20260807 10:50:46.517508 15355 raft_consensus.cc:374] T c9ff5db66b884543b39d73cd3d905c66 P 4ef46d7a4eec4247881e5ae633b95839 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:46.517663 15355 raft_consensus.cc:729] T c9ff5db66b884543b39d73cd3d905c66 P 4ef46d7a4eec4247881e5ae633b95839 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ef46d7a4eec4247881e5ae633b95839, State: Initialized, Role: FOLLOWER
I20260807 10:50:46.518029 15353 consensus_queue.cc:260] T c9ff5db66b884543b39d73cd3d905c66 P 2fcbcf1b2e12435999b48339cbfbd77c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } }
I20260807 10:50:46.518478 15355 consensus_queue.cc:260] T c9ff5db66b884543b39d73cd3d905c66 P 4ef46d7a4eec4247881e5ae633b95839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } }
I20260807 10:50:46.521529 15353 ts_tablet_manager.cc:1428] T c9ff5db66b884543b39d73cd3d905c66 P 2fcbcf1b2e12435999b48339cbfbd77c: Time spent starting tablet: real 0.023s user 0.011s sys 0.012s
I20260807 10:50:46.526149 15354 ts_tablet_manager.cc:1428] T c9ff5db66b884543b39d73cd3d905c66 P cad93d47780840d69dae51ac99bd7bff: Time spent starting tablet: real 0.026s user 0.012s sys 0.012s
I20260807 10:50:46.526674 15313 heartbeater.cc:503] Master 127.3.73.254:42755 was elected leader, sending a full tablet report...
I20260807 10:50:46.526889 15355 ts_tablet_manager.cc:1428] T c9ff5db66b884543b39d73cd3d905c66 P 4ef46d7a4eec4247881e5ae633b95839: Time spent starting tablet: real 0.029s user 0.018s sys 0.011s
I20260807 10:50:46.527015 15105 heartbeater.cc:503] Master 127.3.73.254:42755 was elected leader, sending a full tablet report...
W20260807 10:50:46.533912 15314 tablet.cc:2369] T c9ff5db66b884543b39d73cd3d905c66 P cad93d47780840d69dae51ac99bd7bff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:50:46.558066 15363 raft_consensus.cc:482] T c9ff5db66b884543b39d73cd3d905c66 P cad93d47780840d69dae51ac99bd7bff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 10:50:46.558715 15363 raft_consensus.cc:504] T c9ff5db66b884543b39d73cd3d905c66 P cad93d47780840d69dae51ac99bd7bff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } }
I20260807 10:50:46.561633 15363 leader_election.cc:288] T c9ff5db66b884543b39d73cd3d905c66 P cad93d47780840d69dae51ac99bd7bff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2fcbcf1b2e12435999b48339cbfbd77c (127.3.73.193:43623), 4ef46d7a4eec4247881e5ae633b95839 (127.3.73.194:46501)
W20260807 10:50:46.566747 14898 tablet.cc:2369] T c9ff5db66b884543b39d73cd3d905c66 P 2fcbcf1b2e12435999b48339cbfbd77c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:50:46.570685 15145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:46.562347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:43623 (local address 127.3.73.195:40403)
0807 10:50:46.562586 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:46.562605 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:46.562783 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:46.563485 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:46.563489 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:46.563581 (+ 92us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:50:46.563920 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:46.563933 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.565545 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.565549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:46.567148 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:46.567163 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.569669 (+ 2506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.569675 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:46.570065 (+ 390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:46.570102 (+ 37us) client_negotiation.cc:769] Sending connection context
0807 10:50:46.570215 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 10:50:46.570354 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260807 10:50:46.570935 15345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:46.562074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:40403 (local address 127.3.73.193:43623)
0807 10:50:46.562292 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:46.562297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:46.562699 (+ 402us) server_negotiation.cc:407] Connection header received
0807 10:50:46.563021 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:46.563028 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:46.563117 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:46.563204 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:50:46.564082 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.565313 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.567335 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.569470 (+ 2135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.569753 (+ 283us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:46.569845 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:46.570432 (+ 587us) server_negotiation.cc:299] Negotiation successful
0807 10:50:46.570594 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 10:50:46.572387 14823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9ff5db66b884543b39d73cd3d905c66" candidate_uuid: "cad93d47780840d69dae51ac99bd7bff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" is_pre_election: true
I20260807 10:50:46.573449 14823 raft_consensus.cc:2454] T c9ff5db66b884543b39d73cd3d905c66 P 2fcbcf1b2e12435999b48339cbfbd77c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cad93d47780840d69dae51ac99bd7bff in term 0.
I20260807 10:50:46.574139 15352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:46.563113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:37849 (local address 127.3.73.194:46501)
0807 10:50:46.563370 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:46.563376 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:46.564525 (+ 1149us) server_negotiation.cc:407] Connection header received
0807 10:50:46.564617 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:46.564623 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:46.564704 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:46.564823 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:50:46.565882 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.567512 (+ 1630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.571273 (+ 3761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.572558 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.572837 (+ 279us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:46.572931 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:46.573520 (+ 589us) server_negotiation.cc:299] Negotiation successful
0807 10:50:46.573641 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 10:50:46.574469 15366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:46.562877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:46501 (local address 127.3.73.195:37849)
0807 10:50:46.564388 (+ 1511us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:46.564403 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:46.564523 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:46.564867 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:46.564870 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:46.564890 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:50:46.565676 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:46.565689 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.567720 (+ 2031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.567741 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:46.571025 (+ 3284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:46.571045 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.572977 (+ 1932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.572986 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:46.573256 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:46.573319 (+ 63us) client_negotiation.cc:769] Sending connection context
0807 10:50:46.573462 (+ 143us) client_negotiation.cc:240] Negotiation successful
0807 10:50:46.573647 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1367,"thread_start_us":1053,"threads_started":1}
I20260807 10:50:46.574853 15134 leader_election.cc:302] T c9ff5db66b884543b39d73cd3d905c66 P cad93d47780840d69dae51ac99bd7bff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fcbcf1b2e12435999b48339cbfbd77c, cad93d47780840d69dae51ac99bd7bff; no voters:
I20260807 10:50:46.575575 15363 raft_consensus.cc:2781] T c9ff5db66b884543b39d73cd3d905c66 P cad93d47780840d69dae51ac99bd7bff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 10:50:46.575850 15363 raft_consensus.cc:482] T c9ff5db66b884543b39d73cd3d905c66 P cad93d47780840d69dae51ac99bd7bff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 10:50:46.576006 15363 raft_consensus.cc:3037] T c9ff5db66b884543b39d73cd3d905c66 P cad93d47780840d69dae51ac99bd7bff [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:46.576987 15023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9ff5db66b884543b39d73cd3d905c66" candidate_uuid: "cad93d47780840d69dae51ac99bd7bff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ef46d7a4eec4247881e5ae633b95839" is_pre_election: true
I20260807 10:50:46.577921 15023 raft_consensus.cc:2454] T c9ff5db66b884543b39d73cd3d905c66 P 4ef46d7a4eec4247881e5ae633b95839 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cad93d47780840d69dae51ac99bd7bff in term 0.
I20260807 10:50:46.579147 15363 raft_consensus.cc:504] T c9ff5db66b884543b39d73cd3d905c66 P cad93d47780840d69dae51ac99bd7bff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } }
I20260807 10:50:46.579967 15363 leader_election.cc:288] T c9ff5db66b884543b39d73cd3d905c66 P cad93d47780840d69dae51ac99bd7bff [CANDIDATE]: Term 1 election: Requested vote from peers 2fcbcf1b2e12435999b48339cbfbd77c (127.3.73.193:43623), 4ef46d7a4eec4247881e5ae633b95839 (127.3.73.194:46501)
I20260807 10:50:46.580528 15023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9ff5db66b884543b39d73cd3d905c66" candidate_uuid: "cad93d47780840d69dae51ac99bd7bff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ef46d7a4eec4247881e5ae633b95839"
I20260807 10:50:46.580579 14823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9ff5db66b884543b39d73cd3d905c66" candidate_uuid: "cad93d47780840d69dae51ac99bd7bff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fcbcf1b2e12435999b48339cbfbd77c"
I20260807 10:50:46.580859 14823 raft_consensus.cc:3037] T c9ff5db66b884543b39d73cd3d905c66 P 2fcbcf1b2e12435999b48339cbfbd77c [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:46.580837 15023 raft_consensus.cc:3037] T c9ff5db66b884543b39d73cd3d905c66 P 4ef46d7a4eec4247881e5ae633b95839 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:46.583421 14823 raft_consensus.cc:2454] T c9ff5db66b884543b39d73cd3d905c66 P 2fcbcf1b2e12435999b48339cbfbd77c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cad93d47780840d69dae51ac99bd7bff in term 1.
I20260807 10:50:46.583501 15023 raft_consensus.cc:2454] T c9ff5db66b884543b39d73cd3d905c66 P 4ef46d7a4eec4247881e5ae633b95839 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cad93d47780840d69dae51ac99bd7bff in term 1.
I20260807 10:50:46.584090 15134 leader_election.cc:302] T c9ff5db66b884543b39d73cd3d905c66 P cad93d47780840d69dae51ac99bd7bff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fcbcf1b2e12435999b48339cbfbd77c, cad93d47780840d69dae51ac99bd7bff; no voters:
I20260807 10:50:46.584479 15363 raft_consensus.cc:2781] T c9ff5db66b884543b39d73cd3d905c66 P cad93d47780840d69dae51ac99bd7bff [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:50:46.585125 15363 raft_consensus.cc:686] T c9ff5db66b884543b39d73cd3d905c66 P cad93d47780840d69dae51ac99bd7bff [term 1 LEADER]: Becoming Leader. State: Replica: cad93d47780840d69dae51ac99bd7bff, State: Running, Role: LEADER
I20260807 10:50:46.585752 15363 consensus_queue.cc:237] T c9ff5db66b884543b39d73cd3d905c66 P cad93d47780840d69dae51ac99bd7bff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } }
I20260807 10:50:46.591753 14584 catalog_manager.cc:5495] T c9ff5db66b884543b39d73cd3d905c66 P cad93d47780840d69dae51ac99bd7bff reported cstate change: term changed from 0 to 1, leader changed from <none> to cad93d47780840d69dae51ac99bd7bff (127.3.73.195). New cstate: current_term: 1 leader_uuid: "cad93d47780840d69dae51ac99bd7bff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } health_report { overall_health: UNKNOWN } } }
I20260807 10:50:46.612886 15346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:46.607651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39492 (local address 127.3.73.195:41029)
0807 10:50:46.607839 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:46.607844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:46.607860 (+ 16us) server_negotiation.cc:407] Connection header received
0807 10:50:46.607958 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:46.607962 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:46.608038 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:46.608171 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:50:46.608589 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.609341 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.610773 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.611207 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.611304 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:46.611701 (+ 397us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:50:46.612223 (+ 522us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:50:46.612333 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:46.612468 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 10:50:46.612536 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 10:50:46.620972 15023 raft_consensus.cc:1261] T c9ff5db66b884543b39d73cd3d905c66 P 4ef46d7a4eec4247881e5ae633b95839 [term 1 FOLLOWER]: Refusing update from remote peer cad93d47780840d69dae51ac99bd7bff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:50:46.621081 14823 raft_consensus.cc:1261] T c9ff5db66b884543b39d73cd3d905c66 P 2fcbcf1b2e12435999b48339cbfbd77c [term 1 FOLLOWER]: Refusing update from remote peer cad93d47780840d69dae51ac99bd7bff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:50:46.622071 15368 consensus_queue.cc:1035] T c9ff5db66b884543b39d73cd3d905c66 P cad93d47780840d69dae51ac99bd7bff [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:50:46.622463 15363 consensus_queue.cc:1035] T c9ff5db66b884543b39d73cd3d905c66 P cad93d47780840d69dae51ac99bd7bff [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:50:46.632033 15372 mvcc.cc:205] Tried to move back new op lower bound from 7315864971749502976 to 7315864971620134912. Current Snapshot: MvccSnapshot[applied={T|T < 7315864971749502976}]
I20260807 10:50:46.637071 15373 mvcc.cc:205] Tried to move back new op lower bound from 7315864971749502976 to 7315864971620134912. Current Snapshot: MvccSnapshot[applied={T|T < 7315864971749502976}]
I20260807 10:50:46.647205 14587 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34460:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:50:46.647611 14587 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34460:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:50:46.652926 14587 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 014a586fde6e4cd8be60dd6be039aa30: Sending DeleteTablet for 3 replicas of tablet c9ff5db66b884543b39d73cd3d905c66
I20260807 10:50:46.654510 14793 tablet_service.cc:1514] Processing DeleteTablet for tablet c9ff5db66b884543b39d73cd3d905c66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:50:46 UTC) from {username='oryx', principal='oryx/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:37446
I20260807 10:50:46.654644 15200 tablet_service.cc:1514] Processing DeleteTablet for tablet c9ff5db66b884543b39d73cd3d905c66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:50:46 UTC) from {username='oryx', principal='oryx/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:39488
I20260807 10:50:46.657763 14997 tablet_service.cc:1514] Processing DeleteTablet for tablet c9ff5db66b884543b39d73cd3d905c66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:50:46 UTC) from {username='oryx', principal='oryx/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:57538
I20260807 10:50:46.658447 15383 tablet_replica.cc:330] T c9ff5db66b884543b39d73cd3d905c66 P cad93d47780840d69dae51ac99bd7bff: stopping tablet replica
I20260807 10:50:46.658999 15383 raft_consensus.cc:2229] T c9ff5db66b884543b39d73cd3d905c66 P cad93d47780840d69dae51ac99bd7bff [term 1 LEADER]: Raft consensus shutting down.
I20260807 10:50:46.659514 14587 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34460:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 10:50:46.659832 15383 raft_consensus.cc:2258] T c9ff5db66b884543b39d73cd3d905c66 P cad93d47780840d69dae51ac99bd7bff [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 10:50:46.660487 14587 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 10:50:46.661947 15386 tablet_replica.cc:330] T c9ff5db66b884543b39d73cd3d905c66 P 4ef46d7a4eec4247881e5ae633b95839: stopping tablet replica
I20260807 10:50:46.662473 15383 ts_tablet_manager.cc:1905] T c9ff5db66b884543b39d73cd3d905c66 P cad93d47780840d69dae51ac99bd7bff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:50:46.662901 15386 raft_consensus.cc:2229] T c9ff5db66b884543b39d73cd3d905c66 P 4ef46d7a4eec4247881e5ae633b95839 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:50:46.663391 15386 raft_consensus.cc:2258] T c9ff5db66b884543b39d73cd3d905c66 P 4ef46d7a4eec4247881e5ae633b95839 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:50:46.666775 15386 ts_tablet_manager.cc:1905] T c9ff5db66b884543b39d73cd3d905c66 P 4ef46d7a4eec4247881e5ae633b95839: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:50:46.671089 15200 tablet_service.cc:1467] Processing CreateTablet for tablet 2353bc72c5f949f4b5727c490484d1e8 (DEFAULT_TABLE table=test-table [id=36e3a614cf1845e4a9327544ce10fee8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:50:46.671563 15200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2353bc72c5f949f4b5727c490484d1e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:46.674928 15384 tablet_replica.cc:330] T c9ff5db66b884543b39d73cd3d905c66 P 2fcbcf1b2e12435999b48339cbfbd77c: stopping tablet replica
I20260807 10:50:46.677779 15386 ts_tablet_manager.cc:1918] T c9ff5db66b884543b39d73cd3d905c66 P 4ef46d7a4eec4247881e5ae633b95839: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:50:46.677929 14997 tablet_service.cc:1467] Processing CreateTablet for tablet 2353bc72c5f949f4b5727c490484d1e8 (DEFAULT_TABLE table=test-table [id=36e3a614cf1845e4a9327544ce10fee8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:50:46.678048 15386 log.cc:1196] T c9ff5db66b884543b39d73cd3d905c66 P 4ef46d7a4eec4247881e5ae633b95839: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-1/wal/wals/c9ff5db66b884543b39d73cd3d905c66
I20260807 10:50:46.678449 14997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2353bc72c5f949f4b5727c490484d1e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:46.678912 15386 ts_tablet_manager.cc:1939] T c9ff5db66b884543b39d73cd3d905c66 P 4ef46d7a4eec4247881e5ae633b95839: Deleting consensus metadata
I20260807 10:50:46.680428 15383 ts_tablet_manager.cc:1918] T c9ff5db66b884543b39d73cd3d905c66 P cad93d47780840d69dae51ac99bd7bff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:50:46.680615 15383 log.cc:1196] T c9ff5db66b884543b39d73cd3d905c66 P cad93d47780840d69dae51ac99bd7bff: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-2/wal/wals/c9ff5db66b884543b39d73cd3d905c66
I20260807 10:50:46.681221 15383 ts_tablet_manager.cc:1939] T c9ff5db66b884543b39d73cd3d905c66 P cad93d47780840d69dae51ac99bd7bff: Deleting consensus metadata
I20260807 10:50:46.681561 14571 catalog_manager.cc:4841] TS 4ef46d7a4eec4247881e5ae633b95839 (127.3.73.194:46501): tablet c9ff5db66b884543b39d73cd3d905c66 (table test-table [id=d7f9f101d957493abdcd75df8a506c0e]) successfully deleted
I20260807 10:50:46.682801 14572 catalog_manager.cc:4841] TS cad93d47780840d69dae51ac99bd7bff (127.3.73.195:41029): tablet c9ff5db66b884543b39d73cd3d905c66 (table test-table [id=d7f9f101d957493abdcd75df8a506c0e]) successfully deleted
I20260807 10:50:46.685634 14793 tablet_service.cc:1467] Processing CreateTablet for tablet 2353bc72c5f949f4b5727c490484d1e8 (DEFAULT_TABLE table=test-table [id=36e3a614cf1845e4a9327544ce10fee8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:50:46.687173 14793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2353bc72c5f949f4b5727c490484d1e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:46.694478 15354 tablet_bootstrap.cc:492] T 2353bc72c5f949f4b5727c490484d1e8 P cad93d47780840d69dae51ac99bd7bff: Bootstrap starting.
I20260807 10:50:46.694778 15384 raft_consensus.cc:2229] T c9ff5db66b884543b39d73cd3d905c66 P 2fcbcf1b2e12435999b48339cbfbd77c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:50:46.696532 15354 tablet_bootstrap.cc:654] T 2353bc72c5f949f4b5727c490484d1e8 P cad93d47780840d69dae51ac99bd7bff: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:46.697723 15384 raft_consensus.cc:2258] T c9ff5db66b884543b39d73cd3d905c66 P 2fcbcf1b2e12435999b48339cbfbd77c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:50:46.698370 15355 tablet_bootstrap.cc:492] T 2353bc72c5f949f4b5727c490484d1e8 P 4ef46d7a4eec4247881e5ae633b95839: Bootstrap starting.
I20260807 10:50:46.698506 15353 tablet_bootstrap.cc:492] T 2353bc72c5f949f4b5727c490484d1e8 P 2fcbcf1b2e12435999b48339cbfbd77c: Bootstrap starting.
I20260807 10:50:46.700297 15353 tablet_bootstrap.cc:654] T 2353bc72c5f949f4b5727c490484d1e8 P 2fcbcf1b2e12435999b48339cbfbd77c: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:46.700583 15355 tablet_bootstrap.cc:654] T 2353bc72c5f949f4b5727c490484d1e8 P 4ef46d7a4eec4247881e5ae633b95839: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:46.707340 15355 tablet_bootstrap.cc:492] T 2353bc72c5f949f4b5727c490484d1e8 P 4ef46d7a4eec4247881e5ae633b95839: No bootstrap required, opened a new log
I20260807 10:50:46.707523 15355 ts_tablet_manager.cc:1397] T 2353bc72c5f949f4b5727c490484d1e8 P 4ef46d7a4eec4247881e5ae633b95839: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260807 10:50:46.708488 15355 raft_consensus.cc:348] T 2353bc72c5f949f4b5727c490484d1e8 P 4ef46d7a4eec4247881e5ae633b95839 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } }
I20260807 10:50:46.708766 15355 raft_consensus.cc:374] T 2353bc72c5f949f4b5727c490484d1e8 P 4ef46d7a4eec4247881e5ae633b95839 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:46.708873 15355 raft_consensus.cc:729] T 2353bc72c5f949f4b5727c490484d1e8 P 4ef46d7a4eec4247881e5ae633b95839 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ef46d7a4eec4247881e5ae633b95839, State: Initialized, Role: FOLLOWER
I20260807 10:50:46.709148 15355 consensus_queue.cc:260] T 2353bc72c5f949f4b5727c490484d1e8 P 4ef46d7a4eec4247881e5ae633b95839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } }
I20260807 10:50:46.709862 15355 ts_tablet_manager.cc:1428] T 2353bc72c5f949f4b5727c490484d1e8 P 4ef46d7a4eec4247881e5ae633b95839: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 10:50:46.712538 15354 tablet_bootstrap.cc:492] T 2353bc72c5f949f4b5727c490484d1e8 P cad93d47780840d69dae51ac99bd7bff: No bootstrap required, opened a new log
I20260807 10:50:46.712749 15354 ts_tablet_manager.cc:1397] T 2353bc72c5f949f4b5727c490484d1e8 P cad93d47780840d69dae51ac99bd7bff: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.004s
I20260807 10:50:46.713765 15354 raft_consensus.cc:348] T 2353bc72c5f949f4b5727c490484d1e8 P cad93d47780840d69dae51ac99bd7bff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } }
I20260807 10:50:46.714025 15354 raft_consensus.cc:374] T 2353bc72c5f949f4b5727c490484d1e8 P cad93d47780840d69dae51ac99bd7bff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:46.714144 15354 raft_consensus.cc:729] T 2353bc72c5f949f4b5727c490484d1e8 P cad93d47780840d69dae51ac99bd7bff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cad93d47780840d69dae51ac99bd7bff, State: Initialized, Role: FOLLOWER
I20260807 10:50:46.714463 15354 consensus_queue.cc:260] T 2353bc72c5f949f4b5727c490484d1e8 P cad93d47780840d69dae51ac99bd7bff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } }
I20260807 10:50:46.715214 15354 ts_tablet_manager.cc:1428] T 2353bc72c5f949f4b5727c490484d1e8 P cad93d47780840d69dae51ac99bd7bff: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 10:50:46.715902 15384 ts_tablet_manager.cc:1905] T c9ff5db66b884543b39d73cd3d905c66 P 2fcbcf1b2e12435999b48339cbfbd77c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:50:46.717572 15353 tablet_bootstrap.cc:492] T 2353bc72c5f949f4b5727c490484d1e8 P 2fcbcf1b2e12435999b48339cbfbd77c: No bootstrap required, opened a new log
I20260807 10:50:46.717741 15353 ts_tablet_manager.cc:1397] T 2353bc72c5f949f4b5727c490484d1e8 P 2fcbcf1b2e12435999b48339cbfbd77c: Time spent bootstrapping tablet: real 0.019s user 0.011s sys 0.006s
I20260807 10:50:46.718735 15353 raft_consensus.cc:348] T 2353bc72c5f949f4b5727c490484d1e8 P 2fcbcf1b2e12435999b48339cbfbd77c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } }
I20260807 10:50:46.719005 15353 raft_consensus.cc:374] T 2353bc72c5f949f4b5727c490484d1e8 P 2fcbcf1b2e12435999b48339cbfbd77c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:46.719126 15353 raft_consensus.cc:729] T 2353bc72c5f949f4b5727c490484d1e8 P 2fcbcf1b2e12435999b48339cbfbd77c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fcbcf1b2e12435999b48339cbfbd77c, State: Initialized, Role: FOLLOWER
I20260807 10:50:46.719554 15353 consensus_queue.cc:260] T 2353bc72c5f949f4b5727c490484d1e8 P 2fcbcf1b2e12435999b48339cbfbd77c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } }
I20260807 10:50:46.720463 15353 ts_tablet_manager.cc:1428] T 2353bc72c5f949f4b5727c490484d1e8 P 2fcbcf1b2e12435999b48339cbfbd77c: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260807 10:50:46.725018 15384 ts_tablet_manager.cc:1918] T c9ff5db66b884543b39d73cd3d905c66 P 2fcbcf1b2e12435999b48339cbfbd77c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:50:46.725215 15384 log.cc:1196] T c9ff5db66b884543b39d73cd3d905c66 P 2fcbcf1b2e12435999b48339cbfbd77c: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-0/wal/wals/c9ff5db66b884543b39d73cd3d905c66
I20260807 10:50:46.725740 15384 ts_tablet_manager.cc:1939] T c9ff5db66b884543b39d73cd3d905c66 P 2fcbcf1b2e12435999b48339cbfbd77c: Deleting consensus metadata
I20260807 10:50:46.728139 14571 catalog_manager.cc:4841] TS 2fcbcf1b2e12435999b48339cbfbd77c (127.3.73.193:43623): tablet c9ff5db66b884543b39d73cd3d905c66 (table test-table [id=d7f9f101d957493abdcd75df8a506c0e]) successfully deleted
I20260807 10:50:46.791039 15363 raft_consensus.cc:482] T 2353bc72c5f949f4b5727c490484d1e8 P cad93d47780840d69dae51ac99bd7bff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 10:50:46.791241 15363 raft_consensus.cc:504] T 2353bc72c5f949f4b5727c490484d1e8 P cad93d47780840d69dae51ac99bd7bff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } }
I20260807 10:50:46.792012 15363 leader_election.cc:288] T 2353bc72c5f949f4b5727c490484d1e8 P cad93d47780840d69dae51ac99bd7bff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2fcbcf1b2e12435999b48339cbfbd77c (127.3.73.193:43623), 4ef46d7a4eec4247881e5ae633b95839 (127.3.73.194:46501)
I20260807 10:50:46.792778 15023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2353bc72c5f949f4b5727c490484d1e8" candidate_uuid: "cad93d47780840d69dae51ac99bd7bff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ef46d7a4eec4247881e5ae633b95839" is_pre_election: true
I20260807 10:50:46.792778 14823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2353bc72c5f949f4b5727c490484d1e8" candidate_uuid: "cad93d47780840d69dae51ac99bd7bff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" is_pre_election: true
I20260807 10:50:46.793078 15023 raft_consensus.cc:2454] T 2353bc72c5f949f4b5727c490484d1e8 P 4ef46d7a4eec4247881e5ae633b95839 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cad93d47780840d69dae51ac99bd7bff in term 0.
I20260807 10:50:46.793067 14823 raft_consensus.cc:2454] T 2353bc72c5f949f4b5727c490484d1e8 P 2fcbcf1b2e12435999b48339cbfbd77c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cad93d47780840d69dae51ac99bd7bff in term 0.
I20260807 10:50:46.793601 15134 leader_election.cc:302] T 2353bc72c5f949f4b5727c490484d1e8 P cad93d47780840d69dae51ac99bd7bff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ef46d7a4eec4247881e5ae633b95839, cad93d47780840d69dae51ac99bd7bff; no voters:
I20260807 10:50:46.793959 15363 raft_consensus.cc:2781] T 2353bc72c5f949f4b5727c490484d1e8 P cad93d47780840d69dae51ac99bd7bff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 10:50:46.794106 15363 raft_consensus.cc:482] T 2353bc72c5f949f4b5727c490484d1e8 P cad93d47780840d69dae51ac99bd7bff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 10:50:46.794225 15363 raft_consensus.cc:3037] T 2353bc72c5f949f4b5727c490484d1e8 P cad93d47780840d69dae51ac99bd7bff [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:46.797982 15363 raft_consensus.cc:504] T 2353bc72c5f949f4b5727c490484d1e8 P cad93d47780840d69dae51ac99bd7bff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } }
I20260807 10:50:46.799017 15363 leader_election.cc:288] T 2353bc72c5f949f4b5727c490484d1e8 P cad93d47780840d69dae51ac99bd7bff [CANDIDATE]: Term 1 election: Requested vote from peers 2fcbcf1b2e12435999b48339cbfbd77c (127.3.73.193:43623), 4ef46d7a4eec4247881e5ae633b95839 (127.3.73.194:46501)
I20260807 10:50:46.800544 14823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2353bc72c5f949f4b5727c490484d1e8" candidate_uuid: "cad93d47780840d69dae51ac99bd7bff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fcbcf1b2e12435999b48339cbfbd77c"
I20260807 10:50:46.800544 15023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2353bc72c5f949f4b5727c490484d1e8" candidate_uuid: "cad93d47780840d69dae51ac99bd7bff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ef46d7a4eec4247881e5ae633b95839"
I20260807 10:50:46.800819 14823 raft_consensus.cc:3037] T 2353bc72c5f949f4b5727c490484d1e8 P 2fcbcf1b2e12435999b48339cbfbd77c [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:46.800819 15023 raft_consensus.cc:3037] T 2353bc72c5f949f4b5727c490484d1e8 P 4ef46d7a4eec4247881e5ae633b95839 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:46.804026 15023 raft_consensus.cc:2454] T 2353bc72c5f949f4b5727c490484d1e8 P 4ef46d7a4eec4247881e5ae633b95839 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cad93d47780840d69dae51ac99bd7bff in term 1.
I20260807 10:50:46.804026 14823 raft_consensus.cc:2454] T 2353bc72c5f949f4b5727c490484d1e8 P 2fcbcf1b2e12435999b48339cbfbd77c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cad93d47780840d69dae51ac99bd7bff in term 1.
I20260807 10:50:46.804602 15134 leader_election.cc:302] T 2353bc72c5f949f4b5727c490484d1e8 P cad93d47780840d69dae51ac99bd7bff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fcbcf1b2e12435999b48339cbfbd77c, cad93d47780840d69dae51ac99bd7bff; no voters:
I20260807 10:50:46.805197 15363 raft_consensus.cc:2781] T 2353bc72c5f949f4b5727c490484d1e8 P cad93d47780840d69dae51ac99bd7bff [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:50:46.805397 15363 raft_consensus.cc:686] T 2353bc72c5f949f4b5727c490484d1e8 P cad93d47780840d69dae51ac99bd7bff [term 1 LEADER]: Becoming Leader. State: Replica: cad93d47780840d69dae51ac99bd7bff, State: Running, Role: LEADER
I20260807 10:50:46.805759 15363 consensus_queue.cc:237] T 2353bc72c5f949f4b5727c490484d1e8 P cad93d47780840d69dae51ac99bd7bff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } }
I20260807 10:50:46.809284 14587 catalog_manager.cc:5495] T 2353bc72c5f949f4b5727c490484d1e8 P cad93d47780840d69dae51ac99bd7bff reported cstate change: term changed from 0 to 1, leader changed from <none> to cad93d47780840d69dae51ac99bd7bff (127.3.73.195). New cstate: current_term: 1 leader_uuid: "cad93d47780840d69dae51ac99bd7bff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } health_report { overall_health: HEALTHY } } }
Aug 07 10:50:46 dist-test-slave-x5fp krb5kdc[14524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099844, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.254@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM
I20260807 10:50:46.856518 15404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:46.839951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:42755 (local address 127.0.0.1:34468)
0807 10:50:46.840711 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:46.840756 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:46.840927 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:46.841343 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:46.841348 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:46.842309 (+ 961us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:46.842680 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:46.842696 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.843973 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.843979 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:46.845637 (+ 1658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:46.845653 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.846647 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.846652 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:46.846787 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:46.847520 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:46.847542 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:46.850048 (+ 2506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:46.853877 (+ 3829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:46.853889 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:46.853895 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:46.854393 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:46.854816 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:46.854820 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:46.854822 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:46.854977 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:46.855616 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:46.855623 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:46.855799 (+ 176us) client_negotiation.cc:769] Sending connection context
0807 10:50:46.855940 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 10:50:46.856151 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":418,"threads_started":1}
I20260807 10:50:46.856845 15143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:46.839873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34468 (local address 127.3.73.254:42755)
0807 10:50:46.840197 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:46.840207 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:46.840857 (+ 650us) server_negotiation.cc:407] Connection header received
0807 10:50:46.841056 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:46.841060 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:46.841128 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:46.841264 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:46.842859 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.843799 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.845847 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.846470 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.846601 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:46.850260 (+ 3659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:46.850267 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:46.850272 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:46.850313 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:46.853708 (+ 3395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:46.854535 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:46.854541 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:46.854545 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:46.854698 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:46.855102 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:46.855107 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:46.855111 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:46.855470 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:46.855623 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:46.856089 (+ 466us) server_negotiation.cc:299] Negotiation successful
0807 10:50:46.856353 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148}
I20260807 10:50:46.862684 14587 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:34468:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 10:50:46.863942 14587 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 10:50:46.871754 14793 tablet_service.cc:1467] Processing CreateTablet for tablet a023527ce1544602a45b6f5fddc47a38 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=de04dabe8b0a47a69bb87f4beb96276c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 10:50:46.871838 15200 tablet_service.cc:1467] Processing CreateTablet for tablet a023527ce1544602a45b6f5fddc47a38 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=de04dabe8b0a47a69bb87f4beb96276c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 10:50:46.872452 15200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a023527ce1544602a45b6f5fddc47a38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:46.872463 14793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a023527ce1544602a45b6f5fddc47a38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:46.877686 14997 tablet_service.cc:1467] Processing CreateTablet for tablet a023527ce1544602a45b6f5fddc47a38 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=de04dabe8b0a47a69bb87f4beb96276c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 10:50:46.878335 14997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a023527ce1544602a45b6f5fddc47a38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:46.878531 15354 tablet_bootstrap.cc:492] T a023527ce1544602a45b6f5fddc47a38 P cad93d47780840d69dae51ac99bd7bff: Bootstrap starting.
I20260807 10:50:46.879962 15353 tablet_bootstrap.cc:492] T a023527ce1544602a45b6f5fddc47a38 P 2fcbcf1b2e12435999b48339cbfbd77c: Bootstrap starting.
I20260807 10:50:46.880421 15354 tablet_bootstrap.cc:654] T a023527ce1544602a45b6f5fddc47a38 P cad93d47780840d69dae51ac99bd7bff: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:46.881758 15353 tablet_bootstrap.cc:654] T a023527ce1544602a45b6f5fddc47a38 P 2fcbcf1b2e12435999b48339cbfbd77c: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:46.888917 15355 tablet_bootstrap.cc:492] T a023527ce1544602a45b6f5fddc47a38 P 4ef46d7a4eec4247881e5ae633b95839: Bootstrap starting.
I20260807 10:50:46.891238 15353 tablet_bootstrap.cc:492] T a023527ce1544602a45b6f5fddc47a38 P 2fcbcf1b2e12435999b48339cbfbd77c: No bootstrap required, opened a new log
I20260807 10:50:46.891259 15355 tablet_bootstrap.cc:654] T a023527ce1544602a45b6f5fddc47a38 P 4ef46d7a4eec4247881e5ae633b95839: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:46.891408 15353 ts_tablet_manager.cc:1397] T a023527ce1544602a45b6f5fddc47a38 P 2fcbcf1b2e12435999b48339cbfbd77c: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.006s
I20260807 10:50:46.892297 15354 tablet_bootstrap.cc:492] T a023527ce1544602a45b6f5fddc47a38 P cad93d47780840d69dae51ac99bd7bff: No bootstrap required, opened a new log
I20260807 10:50:46.892153 15353 raft_consensus.cc:348] T a023527ce1544602a45b6f5fddc47a38 P 2fcbcf1b2e12435999b48339cbfbd77c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } }
I20260807 10:50:46.892436 15353 raft_consensus.cc:374] T a023527ce1544602a45b6f5fddc47a38 P 2fcbcf1b2e12435999b48339cbfbd77c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:46.892509 15354 ts_tablet_manager.cc:1397] T a023527ce1544602a45b6f5fddc47a38 P cad93d47780840d69dae51ac99bd7bff: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.012s
I20260807 10:50:46.892529 15353 raft_consensus.cc:729] T a023527ce1544602a45b6f5fddc47a38 P 2fcbcf1b2e12435999b48339cbfbd77c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fcbcf1b2e12435999b48339cbfbd77c, State: Initialized, Role: FOLLOWER
I20260807 10:50:46.892900 15353 consensus_queue.cc:260] T a023527ce1544602a45b6f5fddc47a38 P 2fcbcf1b2e12435999b48339cbfbd77c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } }
I20260807 10:50:46.893394 15364 tablet_replica.cc:439] T a023527ce1544602a45b6f5fddc47a38 P 2fcbcf1b2e12435999b48339cbfbd77c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } } }
I20260807 10:50:46.893741 15364 tablet_replica.cc:442] T a023527ce1544602a45b6f5fddc47a38 P 2fcbcf1b2e12435999b48339cbfbd77c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 10:50:46.893595 15354 raft_consensus.cc:348] T a023527ce1544602a45b6f5fddc47a38 P cad93d47780840d69dae51ac99bd7bff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } }
I20260807 10:50:46.893903 15354 raft_consensus.cc:374] T a023527ce1544602a45b6f5fddc47a38 P cad93d47780840d69dae51ac99bd7bff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:46.894090 15354 raft_consensus.cc:729] T a023527ce1544602a45b6f5fddc47a38 P cad93d47780840d69dae51ac99bd7bff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cad93d47780840d69dae51ac99bd7bff, State: Initialized, Role: FOLLOWER
I20260807 10:50:46.894451 15354 consensus_queue.cc:260] T a023527ce1544602a45b6f5fddc47a38 P cad93d47780840d69dae51ac99bd7bff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } }
I20260807 10:50:46.895064 15353 ts_tablet_manager.cc:1428] T a023527ce1544602a45b6f5fddc47a38 P 2fcbcf1b2e12435999b48339cbfbd77c: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260807 10:50:46.895031 15363 tablet_replica.cc:439] T a023527ce1544602a45b6f5fddc47a38 P cad93d47780840d69dae51ac99bd7bff: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } } }
I20260807 10:50:46.895385 15363 tablet_replica.cc:442] T a023527ce1544602a45b6f5fddc47a38 P cad93d47780840d69dae51ac99bd7bff: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 10:50:46.896028 15354 ts_tablet_manager.cc:1428] T a023527ce1544602a45b6f5fddc47a38 P cad93d47780840d69dae51ac99bd7bff: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 10:50:46.902837 15355 tablet_bootstrap.cc:492] T a023527ce1544602a45b6f5fddc47a38 P 4ef46d7a4eec4247881e5ae633b95839: No bootstrap required, opened a new log
I20260807 10:50:46.903038 15355 ts_tablet_manager.cc:1397] T a023527ce1544602a45b6f5fddc47a38 P 4ef46d7a4eec4247881e5ae633b95839: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.007s
I20260807 10:50:46.903921 15355 raft_consensus.cc:348] T a023527ce1544602a45b6f5fddc47a38 P 4ef46d7a4eec4247881e5ae633b95839 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } }
I20260807 10:50:46.904194 15355 raft_consensus.cc:374] T a023527ce1544602a45b6f5fddc47a38 P 4ef46d7a4eec4247881e5ae633b95839 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:46.904302 15355 raft_consensus.cc:729] T a023527ce1544602a45b6f5fddc47a38 P 4ef46d7a4eec4247881e5ae633b95839 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ef46d7a4eec4247881e5ae633b95839, State: Initialized, Role: FOLLOWER
I20260807 10:50:46.904613 15355 consensus_queue.cc:260] T a023527ce1544602a45b6f5fddc47a38 P 4ef46d7a4eec4247881e5ae633b95839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } }
I20260807 10:50:46.905282 15355 ts_tablet_manager.cc:1428] T a023527ce1544602a45b6f5fddc47a38 P 4ef46d7a4eec4247881e5ae633b95839: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 10:50:46.905211 15365 tablet_replica.cc:439] T a023527ce1544602a45b6f5fddc47a38 P 4ef46d7a4eec4247881e5ae633b95839: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } } }
I20260807 10:50:46.905654 15365 tablet_replica.cc:442] T a023527ce1544602a45b6f5fddc47a38 P 4ef46d7a4eec4247881e5ae633b95839: This TxnStatusTablet replica's current role is: FOLLOWER
W20260807 10:50:46.926178 15106 tablet.cc:2369] T 2353bc72c5f949f4b5727c490484d1e8 P 4ef46d7a4eec4247881e5ae633b95839: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:50:46.926290 15364 raft_consensus.cc:482] T a023527ce1544602a45b6f5fddc47a38 P 2fcbcf1b2e12435999b48339cbfbd77c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 10:50:46.926558 15364 raft_consensus.cc:504] T a023527ce1544602a45b6f5fddc47a38 P 2fcbcf1b2e12435999b48339cbfbd77c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } }
I20260807 10:50:46.928697 15364 leader_election.cc:288] T a023527ce1544602a45b6f5fddc47a38 P 2fcbcf1b2e12435999b48339cbfbd77c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ef46d7a4eec4247881e5ae633b95839 (127.3.73.194:46501), cad93d47780840d69dae51ac99bd7bff (127.3.73.195:41029)
I20260807 10:50:46.938014 15352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:46.929237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:57645 (local address 127.3.73.194:46501)
0807 10:50:46.929448 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:46.929457 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:46.930688 (+ 1231us) server_negotiation.cc:407] Connection header received
0807 10:50:46.930876 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:46.930883 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:46.930994 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:46.931109 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:50:46.931913 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.933188 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.935345 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.936579 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.936785 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:46.936883 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:46.937525 (+ 642us) server_negotiation.cc:299] Negotiation successful
0807 10:50:46.937670 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 10:50:46.938397 15410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:46.929084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:46501 (local address 127.3.73.193:57645)
0807 10:50:46.930551 (+ 1467us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:46.930574 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:46.930717 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:46.931160 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:46.931165 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:46.931196 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:50:46.931716 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:46.931730 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.933374 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.933387 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:46.935163 (+ 1776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:46.935178 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.937061 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.937083 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:46.937309 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:46.937347 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 10:50:46.937482 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 10:50:46.937638 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1330,"mutex_wait_us":649,"thread_start_us":479,"threads_started":1}
I20260807 10:50:46.938776 15346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:46.929389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:53797 (local address 127.3.73.195:41029)
0807 10:50:46.929604 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:46.929612 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:46.930080 (+ 468us) server_negotiation.cc:407] Connection header received
0807 10:50:46.930486 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:46.930491 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:46.930565 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:46.930621 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:50:46.931673 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.933120 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.936018 (+ 2898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.937496 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.937669 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:46.937751 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:46.938338 (+ 587us) server_negotiation.cc:299] Negotiation successful
0807 10:50:46.938474 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 10:50:46.939172 15023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a023527ce1544602a45b6f5fddc47a38" candidate_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ef46d7a4eec4247881e5ae633b95839" is_pre_election: true
I20260807 10:50:46.939483 15409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:46.929082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:41029 (local address 127.3.73.193:53797)
0807 10:50:46.929945 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:46.929975 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:46.930256 (+ 281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:46.930932 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:46.930938 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:46.930968 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:50:46.931441 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:46.931454 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.933345 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.933352 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:46.935798 (+ 2446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:46.935814 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.937918 (+ 2104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.937924 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:46.938136 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:46.938167 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 10:50:46.938982 (+ 815us) client_negotiation.cc:240] Negotiation successful
0807 10:50:46.939139 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":711,"thread_start_us":495,"threads_started":1}
I20260807 10:50:46.939502 15023 raft_consensus.cc:2454] T a023527ce1544602a45b6f5fddc47a38 P 4ef46d7a4eec4247881e5ae633b95839 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fcbcf1b2e12435999b48339cbfbd77c in term 0.
I20260807 10:50:46.940084 14719 leader_election.cc:302] T a023527ce1544602a45b6f5fddc47a38 P 2fcbcf1b2e12435999b48339cbfbd77c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fcbcf1b2e12435999b48339cbfbd77c, 4ef46d7a4eec4247881e5ae633b95839; no voters:
I20260807 10:50:46.940567 15364 raft_consensus.cc:2781] T a023527ce1544602a45b6f5fddc47a38 P 2fcbcf1b2e12435999b48339cbfbd77c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 10:50:46.940418 15237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a023527ce1544602a45b6f5fddc47a38" candidate_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cad93d47780840d69dae51ac99bd7bff" is_pre_election: true
I20260807 10:50:46.940742 15364 raft_consensus.cc:482] T a023527ce1544602a45b6f5fddc47a38 P 2fcbcf1b2e12435999b48339cbfbd77c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 10:50:46.940864 15364 raft_consensus.cc:3037] T a023527ce1544602a45b6f5fddc47a38 P 2fcbcf1b2e12435999b48339cbfbd77c [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:46.941061 15237 raft_consensus.cc:2454] T a023527ce1544602a45b6f5fddc47a38 P cad93d47780840d69dae51ac99bd7bff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fcbcf1b2e12435999b48339cbfbd77c in term 0.
I20260807 10:50:46.943890 15364 raft_consensus.cc:504] T a023527ce1544602a45b6f5fddc47a38 P 2fcbcf1b2e12435999b48339cbfbd77c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } }
I20260807 10:50:46.944751 15364 leader_election.cc:288] T a023527ce1544602a45b6f5fddc47a38 P 2fcbcf1b2e12435999b48339cbfbd77c [CANDIDATE]: Term 1 election: Requested vote from peers 4ef46d7a4eec4247881e5ae633b95839 (127.3.73.194:46501), cad93d47780840d69dae51ac99bd7bff (127.3.73.195:41029)
I20260807 10:50:46.945420 15023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a023527ce1544602a45b6f5fddc47a38" candidate_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ef46d7a4eec4247881e5ae633b95839"
I20260807 10:50:46.945576 15237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a023527ce1544602a45b6f5fddc47a38" candidate_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cad93d47780840d69dae51ac99bd7bff"
I20260807 10:50:46.945737 15023 raft_consensus.cc:3037] T a023527ce1544602a45b6f5fddc47a38 P 4ef46d7a4eec4247881e5ae633b95839 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:46.945811 15237 raft_consensus.cc:3037] T a023527ce1544602a45b6f5fddc47a38 P cad93d47780840d69dae51ac99bd7bff [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:46.948994 15023 raft_consensus.cc:2454] T a023527ce1544602a45b6f5fddc47a38 P 4ef46d7a4eec4247881e5ae633b95839 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fcbcf1b2e12435999b48339cbfbd77c in term 1.
I20260807 10:50:46.949483 15237 raft_consensus.cc:2454] T a023527ce1544602a45b6f5fddc47a38 P cad93d47780840d69dae51ac99bd7bff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fcbcf1b2e12435999b48339cbfbd77c in term 1.
I20260807 10:50:46.949626 14719 leader_election.cc:302] T a023527ce1544602a45b6f5fddc47a38 P 2fcbcf1b2e12435999b48339cbfbd77c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fcbcf1b2e12435999b48339cbfbd77c, 4ef46d7a4eec4247881e5ae633b95839; no voters:
I20260807 10:50:46.950182 15364 raft_consensus.cc:2781] T a023527ce1544602a45b6f5fddc47a38 P 2fcbcf1b2e12435999b48339cbfbd77c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:50:46.950754 15364 raft_consensus.cc:686] T a023527ce1544602a45b6f5fddc47a38 P 2fcbcf1b2e12435999b48339cbfbd77c [term 1 LEADER]: Becoming Leader. State: Replica: 2fcbcf1b2e12435999b48339cbfbd77c, State: Running, Role: LEADER
I20260807 10:50:46.951409 15364 consensus_queue.cc:237] T a023527ce1544602a45b6f5fddc47a38 P 2fcbcf1b2e12435999b48339cbfbd77c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } }
I20260807 10:50:46.953641 15411 tablet_replica.cc:439] T a023527ce1544602a45b6f5fddc47a38 P 2fcbcf1b2e12435999b48339cbfbd77c: TxnStatusTablet state changed. Reason: New leader 2fcbcf1b2e12435999b48339cbfbd77c. Latest consensus state: current_term: 1 leader_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } } }
I20260807 10:50:46.954100 15411 tablet_replica.cc:442] T a023527ce1544602a45b6f5fddc47a38 P 2fcbcf1b2e12435999b48339cbfbd77c: This TxnStatusTablet replica's current role is: LEADER
I20260807 10:50:46.955113 15416 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 10:50:46.955381 15416 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 10:50:46.955905 14587 catalog_manager.cc:5495] T a023527ce1544602a45b6f5fddc47a38 P 2fcbcf1b2e12435999b48339cbfbd77c reported cstate change: term changed from 0 to 1, leader changed from <none> to 2fcbcf1b2e12435999b48339cbfbd77c (127.3.73.193). New cstate: current_term: 1 leader_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } health_report { overall_health: UNKNOWN } } }
I20260807 10:50:46.979684 15404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:46.971276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:43623 (local address 127.0.0.1:37452)
0807 10:50:46.971531 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:46.971555 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:46.971721 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:46.972489 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:46.972495 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:46.972510 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 10:50:46.972777 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:46.972803 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.974464 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.974470 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:46.976239 (+ 1769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:46.976257 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.977482 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.977489 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:46.977708 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:46.977809 (+ 101us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 10:50:46.979017 (+ 1208us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 10:50:46.979047 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 10:50:46.979138 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 10:50:46.979322 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 10:50:46.980499 15345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:46.971470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37452 (local address 127.3.73.193:43623)
0807 10:50:46.971653 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:46.971660 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:46.971683 (+ 23us) server_negotiation.cc:407] Connection header received
0807 10:50:46.971958 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:46.971964 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:46.972119 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:46.972276 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:50:46.972977 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.974276 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.976439 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:46.977192 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:46.977332 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:46.978067 (+ 735us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:50:46.978809 (+ 742us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:50:46.978956 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:46.980053 (+ 1097us) server_negotiation.cc:299] Negotiation successful
0807 10:50:46.980154 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 10:50:46.985823 15023 raft_consensus.cc:1261] T a023527ce1544602a45b6f5fddc47a38 P 4ef46d7a4eec4247881e5ae633b95839 [term 1 FOLLOWER]: Refusing update from remote peer 2fcbcf1b2e12435999b48339cbfbd77c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:50:46.986747 15411 consensus_queue.cc:1035] T a023527ce1544602a45b6f5fddc47a38 P 2fcbcf1b2e12435999b48339cbfbd77c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:50:46.987578 15237 raft_consensus.cc:1261] T a023527ce1544602a45b6f5fddc47a38 P cad93d47780840d69dae51ac99bd7bff [term 1 FOLLOWER]: Refusing update from remote peer 2fcbcf1b2e12435999b48339cbfbd77c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:50:46.988438 15411 consensus_queue.cc:1035] T a023527ce1544602a45b6f5fddc47a38 P 2fcbcf1b2e12435999b48339cbfbd77c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:50:46.992265 15365 tablet_replica.cc:439] T a023527ce1544602a45b6f5fddc47a38 P 4ef46d7a4eec4247881e5ae633b95839: TxnStatusTablet state changed. Reason: New leader 2fcbcf1b2e12435999b48339cbfbd77c. Latest consensus state: current_term: 1 leader_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } } }
I20260807 10:50:46.992609 15365 tablet_replica.cc:442] T a023527ce1544602a45b6f5fddc47a38 P 4ef46d7a4eec4247881e5ae633b95839: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 10:50:46.993150 15411 tablet_replica.cc:439] T a023527ce1544602a45b6f5fddc47a38 P 2fcbcf1b2e12435999b48339cbfbd77c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } } }
I20260807 10:50:46.993383 15411 tablet_replica.cc:442] T a023527ce1544602a45b6f5fddc47a38 P 2fcbcf1b2e12435999b48339cbfbd77c: This TxnStatusTablet replica's current role is: LEADER
I20260807 10:50:46.994616 15375 mvcc.cc:205] Tried to move back new op lower bound from 7315864973244768256 to 7315864973117190144. Current Snapshot: MvccSnapshot[applied={T|T < 7315864973244768256 or (T in {7315864973244768256})}]
I20260807 10:50:46.997677 15365 tablet_replica.cc:439] T a023527ce1544602a45b6f5fddc47a38 P 4ef46d7a4eec4247881e5ae633b95839: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } } }
I20260807 10:50:46.997938 15365 tablet_replica.cc:442] T a023527ce1544602a45b6f5fddc47a38 P 4ef46d7a4eec4247881e5ae633b95839: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 10:50:47.007157 15363 tablet_replica.cc:439] T a023527ce1544602a45b6f5fddc47a38 P cad93d47780840d69dae51ac99bd7bff: TxnStatusTablet state changed. Reason: New leader 2fcbcf1b2e12435999b48339cbfbd77c. Latest consensus state: current_term: 1 leader_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } } }
I20260807 10:50:47.007619 15363 tablet_replica.cc:442] T a023527ce1544602a45b6f5fddc47a38 P cad93d47780840d69dae51ac99bd7bff: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 10:50:47.011430 15364 tablet_replica.cc:439] T a023527ce1544602a45b6f5fddc47a38 P 2fcbcf1b2e12435999b48339cbfbd77c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } } }
I20260807 10:50:47.011885 15364 tablet_replica.cc:442] T a023527ce1544602a45b6f5fddc47a38 P 2fcbcf1b2e12435999b48339cbfbd77c: This TxnStatusTablet replica's current role is: LEADER
I20260807 10:50:47.014946 15363 tablet_replica.cc:439] T a023527ce1544602a45b6f5fddc47a38 P cad93d47780840d69dae51ac99bd7bff: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } } }
I20260807 10:50:47.015280 15363 tablet_replica.cc:442] T a023527ce1544602a45b6f5fddc47a38 P cad93d47780840d69dae51ac99bd7bff: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 10:50:47.031271 15345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:47.024867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:56291 (local address 127.3.73.193:43623)
0807 10:50:47.025085 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:47.025093 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:47.025816 (+ 723us) server_negotiation.cc:407] Connection header received
0807 10:50:47.026062 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:47.026068 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:47.026176 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:47.026281 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:50:47.026897 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:47.027708 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:47.029298 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:47.029756 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:47.029923 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:47.030467 (+ 544us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:50:47.030588 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:50:47.030694 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:47.030840 (+ 146us) server_negotiation.cc:299] Negotiation successful
0807 10:50:47.030931 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 10:50:47.031751 15423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:47.024889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:43623 (local address 127.3.73.195:56291)
0807 10:50:47.025694 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:47.025712 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:47.025900 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:47.026301 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:47.026305 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:47.026317 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 10:50:47.026690 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:47.026699 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:47.027851 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:47.027858 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:47.029134 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:47.029152 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:47.029995 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:47.030002 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:47.030164 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:47.030227 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 10:50:47.030697 (+ 470us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 10:50:47.030711 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 10:50:47.030815 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 10:50:47.030962 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":452,"threads_started":1}
I20260807 10:50:47.043427 14823 raft_consensus.cc:1261] T 2353bc72c5f949f4b5727c490484d1e8 P 2fcbcf1b2e12435999b48339cbfbd77c [term 1 FOLLOWER]: Refusing update from remote peer cad93d47780840d69dae51ac99bd7bff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:50:47.044204 15023 raft_consensus.cc:1261] T 2353bc72c5f949f4b5727c490484d1e8 P 4ef46d7a4eec4247881e5ae633b95839 [term 1 FOLLOWER]: Refusing update from remote peer cad93d47780840d69dae51ac99bd7bff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:50:47.044132 15382 consensus_queue.cc:1035] T 2353bc72c5f949f4b5727c490484d1e8 P cad93d47780840d69dae51ac99bd7bff [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:50:47.044826 15382 consensus_queue.cc:1035] T 2353bc72c5f949f4b5727c490484d1e8 P cad93d47780840d69dae51ac99bd7bff [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:50:47.126346 15346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:47.118540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:46575 (local address 127.3.73.195:41029)
0807 10:50:47.118774 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:47.118781 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:47.119767 (+ 986us) server_negotiation.cc:407] Connection header received
0807 10:50:47.120170 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:47.120176 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:47.120276 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:47.120351 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:50:47.121311 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:47.122499 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:47.124072 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:47.124611 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:47.124821 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:47.125408 (+ 587us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:50:47.125559 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:50:47.125633 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:47.125895 (+ 262us) server_negotiation.cc:299] Negotiation successful
0807 10:50:47.126001 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 10:50:47.127528 15428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:47.118751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:41029 (local address 127.3.73.193:46575)
0807 10:50:47.119632 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:47.119665 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:47.119928 (+ 263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:47.120571 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:47.120575 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:47.120588 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 10:50:47.121101 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:47.121114 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:47.122651 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:47.122654 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:47.123906 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:47.123919 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:47.124862 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:47.124868 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:47.125070 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:47.125168 (+ 98us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 10:50:47.125738 (+ 570us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 10:50:47.125748 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 10:50:47.126900 (+ 1152us) client_negotiation.cc:240] Negotiation successful
0807 10:50:47.127058 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"thread_start_us":428,"threads_started":1}
I20260807 10:50:47.174865 14587 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34460:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:50:47.175180 14587 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34460:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:50:47.178756 14587 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 014a586fde6e4cd8be60dd6be039aa30: Sending DeleteTablet for 3 replicas of tablet 2353bc72c5f949f4b5727c490484d1e8
I20260807 10:50:47.179951 14793 tablet_service.cc:1514] Processing DeleteTablet for tablet 2353bc72c5f949f4b5727c490484d1e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:50:47 UTC) from {username='oryx', principal='oryx/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:37446
I20260807 10:50:47.180114 14997 tablet_service.cc:1514] Processing DeleteTablet for tablet 2353bc72c5f949f4b5727c490484d1e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:50:47 UTC) from {username='oryx', principal='oryx/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:57538
I20260807 10:50:47.180307 15384 tablet_replica.cc:330] T 2353bc72c5f949f4b5727c490484d1e8 P 2fcbcf1b2e12435999b48339cbfbd77c: stopping tablet replica
I20260807 10:50:47.180431 15386 tablet_replica.cc:330] T 2353bc72c5f949f4b5727c490484d1e8 P 4ef46d7a4eec4247881e5ae633b95839: stopping tablet replica
I20260807 10:50:47.180563 15384 raft_consensus.cc:2229] T 2353bc72c5f949f4b5727c490484d1e8 P 2fcbcf1b2e12435999b48339cbfbd77c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:50:47.180646 15386 raft_consensus.cc:2229] T 2353bc72c5f949f4b5727c490484d1e8 P 4ef46d7a4eec4247881e5ae633b95839 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:50:47.180828 15384 raft_consensus.cc:2258] T 2353bc72c5f949f4b5727c490484d1e8 P 2fcbcf1b2e12435999b48339cbfbd77c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:50:47.180871 15386 raft_consensus.cc:2258] T 2353bc72c5f949f4b5727c490484d1e8 P 4ef46d7a4eec4247881e5ae633b95839 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:50:47.181242 15200 tablet_service.cc:1514] Processing DeleteTablet for tablet 2353bc72c5f949f4b5727c490484d1e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:50:47 UTC) from {username='oryx', principal='oryx/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:39488
I20260807 10:50:47.181559 15383 tablet_replica.cc:330] T 2353bc72c5f949f4b5727c490484d1e8 P cad93d47780840d69dae51ac99bd7bff: stopping tablet replica
I20260807 10:50:47.181770 15383 raft_consensus.cc:2229] T 2353bc72c5f949f4b5727c490484d1e8 P cad93d47780840d69dae51ac99bd7bff [term 1 LEADER]: Raft consensus shutting down.
I20260807 10:50:47.182173 15386 ts_tablet_manager.cc:1905] T 2353bc72c5f949f4b5727c490484d1e8 P 4ef46d7a4eec4247881e5ae633b95839: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:50:47.182175 15383 raft_consensus.cc:2258] T 2353bc72c5f949f4b5727c490484d1e8 P cad93d47780840d69dae51ac99bd7bff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:50:47.182597 15384 ts_tablet_manager.cc:1905] T 2353bc72c5f949f4b5727c490484d1e8 P 2fcbcf1b2e12435999b48339cbfbd77c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:50:47.183660 15383 ts_tablet_manager.cc:1905] T 2353bc72c5f949f4b5727c490484d1e8 P cad93d47780840d69dae51ac99bd7bff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:50:47.189756 15384 ts_tablet_manager.cc:1918] T 2353bc72c5f949f4b5727c490484d1e8 P 2fcbcf1b2e12435999b48339cbfbd77c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 10:50:47.189756 15386 ts_tablet_manager.cc:1918] T 2353bc72c5f949f4b5727c490484d1e8 P 4ef46d7a4eec4247881e5ae633b95839: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 10:50:47.189958 15386 log.cc:1196] T 2353bc72c5f949f4b5727c490484d1e8 P 4ef46d7a4eec4247881e5ae633b95839: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-1/wal/wals/2353bc72c5f949f4b5727c490484d1e8
I20260807 10:50:47.189967 15384 log.cc:1196] T 2353bc72c5f949f4b5727c490484d1e8 P 2fcbcf1b2e12435999b48339cbfbd77c: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-0/wal/wals/2353bc72c5f949f4b5727c490484d1e8
I20260807 10:50:47.190459 15386 ts_tablet_manager.cc:1939] T 2353bc72c5f949f4b5727c490484d1e8 P 4ef46d7a4eec4247881e5ae633b95839: Deleting consensus metadata
I20260807 10:50:47.190476 15384 ts_tablet_manager.cc:1939] T 2353bc72c5f949f4b5727c490484d1e8 P 2fcbcf1b2e12435999b48339cbfbd77c: Deleting consensus metadata
I20260807 10:50:47.194214 14571 catalog_manager.cc:4841] TS 2fcbcf1b2e12435999b48339cbfbd77c (127.3.73.193:43623): tablet 2353bc72c5f949f4b5727c490484d1e8 (table test-table [id=36e3a614cf1845e4a9327544ce10fee8]) successfully deleted
I20260807 10:50:47.194516 14571 catalog_manager.cc:4841] TS 4ef46d7a4eec4247881e5ae633b95839 (127.3.73.194:46501): tablet 2353bc72c5f949f4b5727c490484d1e8 (table test-table [id=36e3a614cf1845e4a9327544ce10fee8]) successfully deleted
I20260807 10:50:47.199891 15383 ts_tablet_manager.cc:1918] T 2353bc72c5f949f4b5727c490484d1e8 P cad93d47780840d69dae51ac99bd7bff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 10:50:47.200227 15383 log.cc:1196] T 2353bc72c5f949f4b5727c490484d1e8 P cad93d47780840d69dae51ac99bd7bff: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-2/wal/wals/2353bc72c5f949f4b5727c490484d1e8
I20260807 10:50:47.200871 15383 ts_tablet_manager.cc:1939] T 2353bc72c5f949f4b5727c490484d1e8 P cad93d47780840d69dae51ac99bd7bff: Deleting consensus metadata
I20260807 10:50:47.202365 14572 catalog_manager.cc:4841] TS cad93d47780840d69dae51ac99bd7bff (127.3.73.195:41029): tablet 2353bc72c5f949f4b5727c490484d1e8 (table test-table [id=36e3a614cf1845e4a9327544ce10fee8]) successfully deleted
Aug 07 10:50:47 dist-test-slave-x5fp krb5kdc[14524](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 10:50:47 dist-test-slave-x5fp krb5kdc[14524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:50:47 dist-test-slave-x5fp krb5kdc[14524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 10:50:47.266485 15143 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:50:47.255249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34474 (local address 127.3.73.254:42755)
0807 10:50:47.255495 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:47.255505 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:47.256093 (+ 588us) server_negotiation.cc:407] Connection header received
0807 10:50:47.256329 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:47.256333 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:47.256421 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:47.256536 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:47.258008 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:47.258784 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:47.260359 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:47.260930 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:47.261093 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:47.266275 (+ 5182us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34474: BlockingRecv error: recv got EOF from 127.0.0.1:34474 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 07 10:50:47 dist-test-slave-x5fp krb5kdc[14524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM
I20260807 10:50:47.288489 15143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:47.270671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34476 (local address 127.3.73.254:42755)
0807 10:50:47.270937 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:47.270946 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:47.271490 (+ 544us) server_negotiation.cc:407] Connection header received
0807 10:50:47.271710 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:47.271716 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:47.271796 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:47.271906 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:47.273290 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:47.274050 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:47.275341 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:47.275804 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:47.275950 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:47.280257 (+ 4307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:47.280265 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:47.280269 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:47.280327 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:47.284702 (+ 4375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:47.285741 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:47.285748 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:47.285752 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:47.285948 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:47.286464 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:47.286470 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:47.286474 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:47.286964 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:47.287275 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:47.287652 (+ 377us) server_negotiation.cc:299] Negotiation successful
0807 10:50:47.287935 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260807 10:50:47.294466 14587 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34476:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 10:50:47.295266 14587 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 10:50:47.303740 14997 tablet_service.cc:1467] Processing CreateTablet for tablet 60bfd7d9f04847d3bfb9c07e18c38d67 (DEFAULT_TABLE table=test-table [id=6b047b0490044ff6b03874de72c1e325]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:50:47.303738 14793 tablet_service.cc:1467] Processing CreateTablet for tablet 60bfd7d9f04847d3bfb9c07e18c38d67 (DEFAULT_TABLE table=test-table [id=6b047b0490044ff6b03874de72c1e325]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:50:47.303992 15200 tablet_service.cc:1467] Processing CreateTablet for tablet 60bfd7d9f04847d3bfb9c07e18c38d67 (DEFAULT_TABLE table=test-table [id=6b047b0490044ff6b03874de72c1e325]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:50:47.304248 14997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60bfd7d9f04847d3bfb9c07e18c38d67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:47.304250 14793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60bfd7d9f04847d3bfb9c07e18c38d67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:47.304422 15200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60bfd7d9f04847d3bfb9c07e18c38d67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:47.310956 15355 tablet_bootstrap.cc:492] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 4ef46d7a4eec4247881e5ae633b95839: Bootstrap starting.
I20260807 10:50:47.312688 15355 tablet_bootstrap.cc:654] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 4ef46d7a4eec4247881e5ae633b95839: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:47.313831 15353 tablet_bootstrap.cc:492] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 2fcbcf1b2e12435999b48339cbfbd77c: Bootstrap starting.
I20260807 10:50:47.315574 15353 tablet_bootstrap.cc:654] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 2fcbcf1b2e12435999b48339cbfbd77c: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:47.316636 15354 tablet_bootstrap.cc:492] T 60bfd7d9f04847d3bfb9c07e18c38d67 P cad93d47780840d69dae51ac99bd7bff: Bootstrap starting.
I20260807 10:50:47.318252 15354 tablet_bootstrap.cc:654] T 60bfd7d9f04847d3bfb9c07e18c38d67 P cad93d47780840d69dae51ac99bd7bff: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:47.320618 15355 tablet_bootstrap.cc:492] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 4ef46d7a4eec4247881e5ae633b95839: No bootstrap required, opened a new log
I20260807 10:50:47.321461 15354 tablet_bootstrap.cc:492] T 60bfd7d9f04847d3bfb9c07e18c38d67 P cad93d47780840d69dae51ac99bd7bff: No bootstrap required, opened a new log
I20260807 10:50:47.321614 15354 ts_tablet_manager.cc:1397] T 60bfd7d9f04847d3bfb9c07e18c38d67 P cad93d47780840d69dae51ac99bd7bff: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 10:50:47.322037 15355 ts_tablet_manager.cc:1397] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 4ef46d7a4eec4247881e5ae633b95839: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
I20260807 10:50:47.322369 15354 raft_consensus.cc:348] T 60bfd7d9f04847d3bfb9c07e18c38d67 P cad93d47780840d69dae51ac99bd7bff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } }
I20260807 10:50:47.322655 15354 raft_consensus.cc:374] T 60bfd7d9f04847d3bfb9c07e18c38d67 P cad93d47780840d69dae51ac99bd7bff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:47.322757 15354 raft_consensus.cc:729] T 60bfd7d9f04847d3bfb9c07e18c38d67 P cad93d47780840d69dae51ac99bd7bff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cad93d47780840d69dae51ac99bd7bff, State: Initialized, Role: FOLLOWER
I20260807 10:50:47.322988 15354 consensus_queue.cc:260] T 60bfd7d9f04847d3bfb9c07e18c38d67 P cad93d47780840d69dae51ac99bd7bff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } }
I20260807 10:50:47.323310 15355 raft_consensus.cc:348] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 4ef46d7a4eec4247881e5ae633b95839 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } }
I20260807 10:50:47.323623 15355 raft_consensus.cc:374] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 4ef46d7a4eec4247881e5ae633b95839 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:47.324141 15354 ts_tablet_manager.cc:1428] T 60bfd7d9f04847d3bfb9c07e18c38d67 P cad93d47780840d69dae51ac99bd7bff: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260807 10:50:47.325052 15353 tablet_bootstrap.cc:492] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 2fcbcf1b2e12435999b48339cbfbd77c: No bootstrap required, opened a new log
I20260807 10:50:47.325204 15353 ts_tablet_manager.cc:1397] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 2fcbcf1b2e12435999b48339cbfbd77c: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.002s
I20260807 10:50:47.325979 15353 raft_consensus.cc:348] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 2fcbcf1b2e12435999b48339cbfbd77c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } }
I20260807 10:50:47.326284 15353 raft_consensus.cc:374] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 2fcbcf1b2e12435999b48339cbfbd77c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:47.326373 15353 raft_consensus.cc:729] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 2fcbcf1b2e12435999b48339cbfbd77c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fcbcf1b2e12435999b48339cbfbd77c, State: Initialized, Role: FOLLOWER
I20260807 10:50:47.326615 15353 consensus_queue.cc:260] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 2fcbcf1b2e12435999b48339cbfbd77c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } }
I20260807 10:50:47.328109 15353 ts_tablet_manager.cc:1428] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 2fcbcf1b2e12435999b48339cbfbd77c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 10:50:47.323762 15355 raft_consensus.cc:729] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 4ef46d7a4eec4247881e5ae633b95839 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ef46d7a4eec4247881e5ae633b95839, State: Initialized, Role: FOLLOWER
I20260807 10:50:47.329594 15355 consensus_queue.cc:260] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 4ef46d7a4eec4247881e5ae633b95839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } }
I20260807 10:50:47.330173 15355 ts_tablet_manager.cc:1428] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 4ef46d7a4eec4247881e5ae633b95839: Time spent starting tablet: real 0.008s user 0.001s sys 0.000s
I20260807 10:50:47.401002 15363 raft_consensus.cc:482] T 60bfd7d9f04847d3bfb9c07e18c38d67 P cad93d47780840d69dae51ac99bd7bff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 10:50:47.401243 15363 raft_consensus.cc:504] T 60bfd7d9f04847d3bfb9c07e18c38d67 P cad93d47780840d69dae51ac99bd7bff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } }
I20260807 10:50:47.401930 15363 leader_election.cc:288] T 60bfd7d9f04847d3bfb9c07e18c38d67 P cad93d47780840d69dae51ac99bd7bff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2fcbcf1b2e12435999b48339cbfbd77c (127.3.73.193:43623), 4ef46d7a4eec4247881e5ae633b95839 (127.3.73.194:46501)
I20260807 10:50:47.402765 14822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60bfd7d9f04847d3bfb9c07e18c38d67" candidate_uuid: "cad93d47780840d69dae51ac99bd7bff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" is_pre_election: true
I20260807 10:50:47.402755 15022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60bfd7d9f04847d3bfb9c07e18c38d67" candidate_uuid: "cad93d47780840d69dae51ac99bd7bff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ef46d7a4eec4247881e5ae633b95839" is_pre_election: true
I20260807 10:50:47.403056 15022 raft_consensus.cc:2454] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 4ef46d7a4eec4247881e5ae633b95839 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cad93d47780840d69dae51ac99bd7bff in term 0.
I20260807 10:50:47.403059 14822 raft_consensus.cc:2454] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 2fcbcf1b2e12435999b48339cbfbd77c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cad93d47780840d69dae51ac99bd7bff in term 0.
I20260807 10:50:47.403558 15134 leader_election.cc:302] T 60bfd7d9f04847d3bfb9c07e18c38d67 P cad93d47780840d69dae51ac99bd7bff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ef46d7a4eec4247881e5ae633b95839, cad93d47780840d69dae51ac99bd7bff; no voters:
I20260807 10:50:47.403901 15363 raft_consensus.cc:2781] T 60bfd7d9f04847d3bfb9c07e18c38d67 P cad93d47780840d69dae51ac99bd7bff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 10:50:47.404011 15363 raft_consensus.cc:482] T 60bfd7d9f04847d3bfb9c07e18c38d67 P cad93d47780840d69dae51ac99bd7bff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 10:50:47.404129 15363 raft_consensus.cc:3037] T 60bfd7d9f04847d3bfb9c07e18c38d67 P cad93d47780840d69dae51ac99bd7bff [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:47.406724 15363 raft_consensus.cc:504] T 60bfd7d9f04847d3bfb9c07e18c38d67 P cad93d47780840d69dae51ac99bd7bff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } }
I20260807 10:50:47.407384 15363 leader_election.cc:288] T 60bfd7d9f04847d3bfb9c07e18c38d67 P cad93d47780840d69dae51ac99bd7bff [CANDIDATE]: Term 1 election: Requested vote from peers 2fcbcf1b2e12435999b48339cbfbd77c (127.3.73.193:43623), 4ef46d7a4eec4247881e5ae633b95839 (127.3.73.194:46501)
I20260807 10:50:47.407898 14822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60bfd7d9f04847d3bfb9c07e18c38d67" candidate_uuid: "cad93d47780840d69dae51ac99bd7bff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fcbcf1b2e12435999b48339cbfbd77c"
I20260807 10:50:47.407999 15022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60bfd7d9f04847d3bfb9c07e18c38d67" candidate_uuid: "cad93d47780840d69dae51ac99bd7bff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ef46d7a4eec4247881e5ae633b95839"
I20260807 10:50:47.408128 14822 raft_consensus.cc:3037] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 2fcbcf1b2e12435999b48339cbfbd77c [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:47.408270 15022 raft_consensus.cc:3037] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 4ef46d7a4eec4247881e5ae633b95839 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:47.411077 14822 raft_consensus.cc:2454] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 2fcbcf1b2e12435999b48339cbfbd77c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cad93d47780840d69dae51ac99bd7bff in term 1.
I20260807 10:50:47.411547 15134 leader_election.cc:302] T 60bfd7d9f04847d3bfb9c07e18c38d67 P cad93d47780840d69dae51ac99bd7bff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fcbcf1b2e12435999b48339cbfbd77c, cad93d47780840d69dae51ac99bd7bff; no voters:
I20260807 10:50:47.411538 15022 raft_consensus.cc:2454] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 4ef46d7a4eec4247881e5ae633b95839 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cad93d47780840d69dae51ac99bd7bff in term 1.
I20260807 10:50:47.411991 15363 raft_consensus.cc:2781] T 60bfd7d9f04847d3bfb9c07e18c38d67 P cad93d47780840d69dae51ac99bd7bff [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:50:47.412175 15363 raft_consensus.cc:686] T 60bfd7d9f04847d3bfb9c07e18c38d67 P cad93d47780840d69dae51ac99bd7bff [term 1 LEADER]: Becoming Leader. State: Replica: cad93d47780840d69dae51ac99bd7bff, State: Running, Role: LEADER
I20260807 10:50:47.412453 15363 consensus_queue.cc:237] T 60bfd7d9f04847d3bfb9c07e18c38d67 P cad93d47780840d69dae51ac99bd7bff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } }
I20260807 10:50:47.415516 14584 catalog_manager.cc:5495] T 60bfd7d9f04847d3bfb9c07e18c38d67 P cad93d47780840d69dae51ac99bd7bff reported cstate change: term changed from 0 to 1, leader changed from <none> to cad93d47780840d69dae51ac99bd7bff (127.3.73.195). New cstate: current_term: 1 leader_uuid: "cad93d47780840d69dae51ac99bd7bff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } health_report { overall_health: HEALTHY } } }
I20260807 10:50:47.444172 15346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:47.437787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39500 (local address 127.3.73.195:41029)
0807 10:50:47.438026 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:47.438034 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:47.438127 (+ 93us) server_negotiation.cc:407] Connection header received
0807 10:50:47.438388 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:47.438393 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:47.438469 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:47.438575 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:50:47.439052 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:47.440235 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:47.441710 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:47.442407 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:47.442556 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:47.443226 (+ 670us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:50:47.443385 (+ 159us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:50:47.443506 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:47.443737 (+ 231us) server_negotiation.cc:299] Negotiation successful
0807 10:50:47.443832 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260807 10:50:47.447570 14822 raft_consensus.cc:1261] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 2fcbcf1b2e12435999b48339cbfbd77c [term 1 FOLLOWER]: Refusing update from remote peer cad93d47780840d69dae51ac99bd7bff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:50:47.447628 15022 raft_consensus.cc:1261] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 4ef46d7a4eec4247881e5ae633b95839 [term 1 FOLLOWER]: Refusing update from remote peer cad93d47780840d69dae51ac99bd7bff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:50:47.448323 15368 consensus_queue.cc:1035] T 60bfd7d9f04847d3bfb9c07e18c38d67 P cad93d47780840d69dae51ac99bd7bff [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:50:47.448866 15363 consensus_queue.cc:1035] T 60bfd7d9f04847d3bfb9c07e18c38d67 P cad93d47780840d69dae51ac99bd7bff [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:50:47.467015 14584 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:50:47.467345 14584 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:50:47.470739 14584 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 014a586fde6e4cd8be60dd6be039aa30: Sending DeleteTablet for 3 replicas of tablet 60bfd7d9f04847d3bfb9c07e18c38d67
I20260807 10:50:47.471784 14793 tablet_service.cc:1514] Processing DeleteTablet for tablet 60bfd7d9f04847d3bfb9c07e18c38d67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:50:47 UTC) from {username='oryx', principal='oryx/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:37446
I20260807 10:50:47.471853 14997 tablet_service.cc:1514] Processing DeleteTablet for tablet 60bfd7d9f04847d3bfb9c07e18c38d67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:50:47 UTC) from {username='oryx', principal='oryx/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:57538
I20260807 10:50:47.472165 15386 tablet_replica.cc:330] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 4ef46d7a4eec4247881e5ae633b95839: stopping tablet replica
I20260807 10:50:47.472175 15200 tablet_service.cc:1514] Processing DeleteTablet for tablet 60bfd7d9f04847d3bfb9c07e18c38d67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:50:47 UTC) from {username='oryx', principal='oryx/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:39488
I20260807 10:50:47.472381 15386 raft_consensus.cc:2229] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 4ef46d7a4eec4247881e5ae633b95839 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:50:47.472534 15383 tablet_replica.cc:330] T 60bfd7d9f04847d3bfb9c07e18c38d67 P cad93d47780840d69dae51ac99bd7bff: stopping tablet replica
I20260807 10:50:47.472571 15384 tablet_replica.cc:330] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 2fcbcf1b2e12435999b48339cbfbd77c: stopping tablet replica
I20260807 10:50:47.472666 15386 raft_consensus.cc:2258] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 4ef46d7a4eec4247881e5ae633b95839 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:50:47.472749 15383 raft_consensus.cc:2229] T 60bfd7d9f04847d3bfb9c07e18c38d67 P cad93d47780840d69dae51ac99bd7bff [term 1 LEADER]: Raft consensus shutting down.
I20260807 10:50:47.472812 15384 raft_consensus.cc:2229] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 2fcbcf1b2e12435999b48339cbfbd77c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:50:47.473100 15383 raft_consensus.cc:2258] T 60bfd7d9f04847d3bfb9c07e18c38d67 P cad93d47780840d69dae51ac99bd7bff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:50:47.473083 15384 raft_consensus.cc:2258] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 2fcbcf1b2e12435999b48339cbfbd77c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:50:47.474150 15386 ts_tablet_manager.cc:1905] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 4ef46d7a4eec4247881e5ae633b95839: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:50:47.474948 14584 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34476:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 10:50:47.475564 15384 ts_tablet_manager.cc:1905] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 2fcbcf1b2e12435999b48339cbfbd77c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 10:50:47.475862 14584 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 10:50:47.477676 15383 ts_tablet_manager.cc:1905] T 60bfd7d9f04847d3bfb9c07e18c38d67 P cad93d47780840d69dae51ac99bd7bff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:50:47.484501 14793 tablet_service.cc:1467] Processing CreateTablet for tablet dfaff5775d734ae7bc98cdc86552411b (DEFAULT_TABLE table=test-table [id=40acc189baf44023892612549bb7da4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:50:47.484941 15200 tablet_service.cc:1467] Processing CreateTablet for tablet dfaff5775d734ae7bc98cdc86552411b (DEFAULT_TABLE table=test-table [id=40acc189baf44023892612549bb7da4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:50:47.485364 15200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfaff5775d734ae7bc98cdc86552411b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:47.486676 14793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfaff5775d734ae7bc98cdc86552411b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:47.489917 15386 ts_tablet_manager.cc:1918] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 4ef46d7a4eec4247881e5ae633b95839: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:50:47.490072 15386 log.cc:1196] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 4ef46d7a4eec4247881e5ae633b95839: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-1/wal/wals/60bfd7d9f04847d3bfb9c07e18c38d67
I20260807 10:50:47.490545 15386 ts_tablet_manager.cc:1939] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 4ef46d7a4eec4247881e5ae633b95839: Deleting consensus metadata
I20260807 10:50:47.491806 14571 catalog_manager.cc:4841] TS 4ef46d7a4eec4247881e5ae633b95839 (127.3.73.194:46501): tablet 60bfd7d9f04847d3bfb9c07e18c38d67 (table test-table [id=6b047b0490044ff6b03874de72c1e325]) successfully deleted
I20260807 10:50:47.492044 14997 tablet_service.cc:1467] Processing CreateTablet for tablet dfaff5775d734ae7bc98cdc86552411b (DEFAULT_TABLE table=test-table [id=40acc189baf44023892612549bb7da4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:50:47.492534 14997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfaff5775d734ae7bc98cdc86552411b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:50:47.493850 15353 tablet_bootstrap.cc:492] T dfaff5775d734ae7bc98cdc86552411b P 2fcbcf1b2e12435999b48339cbfbd77c: Bootstrap starting.
I20260807 10:50:47.494643 15383 ts_tablet_manager.cc:1918] T 60bfd7d9f04847d3bfb9c07e18c38d67 P cad93d47780840d69dae51ac99bd7bff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:50:47.495417 15383 log.cc:1196] T 60bfd7d9f04847d3bfb9c07e18c38d67 P cad93d47780840d69dae51ac99bd7bff: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-2/wal/wals/60bfd7d9f04847d3bfb9c07e18c38d67
I20260807 10:50:47.495827 15353 tablet_bootstrap.cc:654] T dfaff5775d734ae7bc98cdc86552411b P 2fcbcf1b2e12435999b48339cbfbd77c: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:47.495909 15383 ts_tablet_manager.cc:1939] T 60bfd7d9f04847d3bfb9c07e18c38d67 P cad93d47780840d69dae51ac99bd7bff: Deleting consensus metadata
I20260807 10:50:47.496243 15354 tablet_bootstrap.cc:492] T dfaff5775d734ae7bc98cdc86552411b P cad93d47780840d69dae51ac99bd7bff: Bootstrap starting.
I20260807 10:50:47.497164 14572 catalog_manager.cc:4841] TS cad93d47780840d69dae51ac99bd7bff (127.3.73.195:41029): tablet 60bfd7d9f04847d3bfb9c07e18c38d67 (table test-table [id=6b047b0490044ff6b03874de72c1e325]) successfully deleted
I20260807 10:50:47.497781 15384 ts_tablet_manager.cc:1918] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 2fcbcf1b2e12435999b48339cbfbd77c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:50:47.497946 15384 log.cc:1196] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 2fcbcf1b2e12435999b48339cbfbd77c: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-0/wal/wals/60bfd7d9f04847d3bfb9c07e18c38d67
I20260807 10:50:47.498371 15354 tablet_bootstrap.cc:654] T dfaff5775d734ae7bc98cdc86552411b P cad93d47780840d69dae51ac99bd7bff: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:47.498646 15384 ts_tablet_manager.cc:1939] T 60bfd7d9f04847d3bfb9c07e18c38d67 P 2fcbcf1b2e12435999b48339cbfbd77c: Deleting consensus metadata
I20260807 10:50:47.501614 14571 catalog_manager.cc:4841] TS 2fcbcf1b2e12435999b48339cbfbd77c (127.3.73.193:43623): tablet 60bfd7d9f04847d3bfb9c07e18c38d67 (table test-table [id=6b047b0490044ff6b03874de72c1e325]) successfully deleted
I20260807 10:50:47.504423 15355 tablet_bootstrap.cc:492] T dfaff5775d734ae7bc98cdc86552411b P 4ef46d7a4eec4247881e5ae633b95839: Bootstrap starting.
I20260807 10:50:47.506594 15355 tablet_bootstrap.cc:654] T dfaff5775d734ae7bc98cdc86552411b P 4ef46d7a4eec4247881e5ae633b95839: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:47.506991 15353 tablet_bootstrap.cc:492] T dfaff5775d734ae7bc98cdc86552411b P 2fcbcf1b2e12435999b48339cbfbd77c: No bootstrap required, opened a new log
I20260807 10:50:47.507341 15353 ts_tablet_manager.cc:1397] T dfaff5775d734ae7bc98cdc86552411b P 2fcbcf1b2e12435999b48339cbfbd77c: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.002s
I20260807 10:50:47.508348 15353 raft_consensus.cc:348] T dfaff5775d734ae7bc98cdc86552411b P 2fcbcf1b2e12435999b48339cbfbd77c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } }
I20260807 10:50:47.508672 15353 raft_consensus.cc:374] T dfaff5775d734ae7bc98cdc86552411b P 2fcbcf1b2e12435999b48339cbfbd77c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:47.508828 15353 raft_consensus.cc:729] T dfaff5775d734ae7bc98cdc86552411b P 2fcbcf1b2e12435999b48339cbfbd77c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fcbcf1b2e12435999b48339cbfbd77c, State: Initialized, Role: FOLLOWER
I20260807 10:50:47.508875 15355 tablet_bootstrap.cc:492] T dfaff5775d734ae7bc98cdc86552411b P 4ef46d7a4eec4247881e5ae633b95839: No bootstrap required, opened a new log
I20260807 10:50:47.509374 15355 ts_tablet_manager.cc:1397] T dfaff5775d734ae7bc98cdc86552411b P 4ef46d7a4eec4247881e5ae633b95839: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260807 10:50:47.509436 15353 consensus_queue.cc:260] T dfaff5775d734ae7bc98cdc86552411b P 2fcbcf1b2e12435999b48339cbfbd77c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } }
I20260807 10:50:47.510615 15355 raft_consensus.cc:348] T dfaff5775d734ae7bc98cdc86552411b P 4ef46d7a4eec4247881e5ae633b95839 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } }
I20260807 10:50:47.510982 15355 raft_consensus.cc:374] T dfaff5775d734ae7bc98cdc86552411b P 4ef46d7a4eec4247881e5ae633b95839 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:47.511034 15353 ts_tablet_manager.cc:1428] T dfaff5775d734ae7bc98cdc86552411b P 2fcbcf1b2e12435999b48339cbfbd77c: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260807 10:50:47.511075 15355 raft_consensus.cc:729] T dfaff5775d734ae7bc98cdc86552411b P 4ef46d7a4eec4247881e5ae633b95839 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ef46d7a4eec4247881e5ae633b95839, State: Initialized, Role: FOLLOWER
I20260807 10:50:47.511372 15355 consensus_queue.cc:260] T dfaff5775d734ae7bc98cdc86552411b P 4ef46d7a4eec4247881e5ae633b95839 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } }
I20260807 10:50:47.512051 15355 ts_tablet_manager.cc:1428] T dfaff5775d734ae7bc98cdc86552411b P 4ef46d7a4eec4247881e5ae633b95839: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 10:50:47.513175 15354 tablet_bootstrap.cc:492] T dfaff5775d734ae7bc98cdc86552411b P cad93d47780840d69dae51ac99bd7bff: No bootstrap required, opened a new log
I20260807 10:50:47.513383 15354 ts_tablet_manager.cc:1397] T dfaff5775d734ae7bc98cdc86552411b P cad93d47780840d69dae51ac99bd7bff: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.001s
I20260807 10:50:47.515168 15354 raft_consensus.cc:348] T dfaff5775d734ae7bc98cdc86552411b P cad93d47780840d69dae51ac99bd7bff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } }
I20260807 10:50:47.515445 15354 raft_consensus.cc:374] T dfaff5775d734ae7bc98cdc86552411b P cad93d47780840d69dae51ac99bd7bff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:47.515542 15354 raft_consensus.cc:729] T dfaff5775d734ae7bc98cdc86552411b P cad93d47780840d69dae51ac99bd7bff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cad93d47780840d69dae51ac99bd7bff, State: Initialized, Role: FOLLOWER
I20260807 10:50:47.515815 15354 consensus_queue.cc:260] T dfaff5775d734ae7bc98cdc86552411b P cad93d47780840d69dae51ac99bd7bff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } }
I20260807 10:50:47.516626 15354 ts_tablet_manager.cc:1428] T dfaff5775d734ae7bc98cdc86552411b P cad93d47780840d69dae51ac99bd7bff: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260807 10:50:47.634761 15424 raft_consensus.cc:482] T dfaff5775d734ae7bc98cdc86552411b P 2fcbcf1b2e12435999b48339cbfbd77c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 10:50:47.634974 15424 raft_consensus.cc:504] T dfaff5775d734ae7bc98cdc86552411b P 2fcbcf1b2e12435999b48339cbfbd77c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } }
I20260807 10:50:47.635766 15424 leader_election.cc:288] T dfaff5775d734ae7bc98cdc86552411b P 2fcbcf1b2e12435999b48339cbfbd77c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ef46d7a4eec4247881e5ae633b95839 (127.3.73.194:46501), cad93d47780840d69dae51ac99bd7bff (127.3.73.195:41029)
I20260807 10:50:47.636199 15022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfaff5775d734ae7bc98cdc86552411b" candidate_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ef46d7a4eec4247881e5ae633b95839" is_pre_election: true
I20260807 10:50:47.636317 15235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfaff5775d734ae7bc98cdc86552411b" candidate_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cad93d47780840d69dae51ac99bd7bff" is_pre_election: true
I20260807 10:50:47.636480 15022 raft_consensus.cc:2454] T dfaff5775d734ae7bc98cdc86552411b P 4ef46d7a4eec4247881e5ae633b95839 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fcbcf1b2e12435999b48339cbfbd77c in term 0.
I20260807 10:50:47.636587 15235 raft_consensus.cc:2454] T dfaff5775d734ae7bc98cdc86552411b P cad93d47780840d69dae51ac99bd7bff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2fcbcf1b2e12435999b48339cbfbd77c in term 0.
I20260807 10:50:47.637048 14719 leader_election.cc:302] T dfaff5775d734ae7bc98cdc86552411b P 2fcbcf1b2e12435999b48339cbfbd77c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2fcbcf1b2e12435999b48339cbfbd77c, 4ef46d7a4eec4247881e5ae633b95839, cad93d47780840d69dae51ac99bd7bff; no voters:
I20260807 10:50:47.637362 15424 raft_consensus.cc:2781] T dfaff5775d734ae7bc98cdc86552411b P 2fcbcf1b2e12435999b48339cbfbd77c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 10:50:47.637518 15424 raft_consensus.cc:482] T dfaff5775d734ae7bc98cdc86552411b P 2fcbcf1b2e12435999b48339cbfbd77c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 10:50:47.637641 15424 raft_consensus.cc:3037] T dfaff5775d734ae7bc98cdc86552411b P 2fcbcf1b2e12435999b48339cbfbd77c [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:47.639658 15424 raft_consensus.cc:504] T dfaff5775d734ae7bc98cdc86552411b P 2fcbcf1b2e12435999b48339cbfbd77c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } }
I20260807 10:50:47.640251 15424 leader_election.cc:288] T dfaff5775d734ae7bc98cdc86552411b P 2fcbcf1b2e12435999b48339cbfbd77c [CANDIDATE]: Term 1 election: Requested vote from peers 4ef46d7a4eec4247881e5ae633b95839 (127.3.73.194:46501), cad93d47780840d69dae51ac99bd7bff (127.3.73.195:41029)
I20260807 10:50:47.640758 15022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfaff5775d734ae7bc98cdc86552411b" candidate_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ef46d7a4eec4247881e5ae633b95839"
I20260807 10:50:47.640889 15235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfaff5775d734ae7bc98cdc86552411b" candidate_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cad93d47780840d69dae51ac99bd7bff"
I20260807 10:50:47.640980 15022 raft_consensus.cc:3037] T dfaff5775d734ae7bc98cdc86552411b P 4ef46d7a4eec4247881e5ae633b95839 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:47.641117 15235 raft_consensus.cc:3037] T dfaff5775d734ae7bc98cdc86552411b P cad93d47780840d69dae51ac99bd7bff [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:47.644119 15022 raft_consensus.cc:2454] T dfaff5775d734ae7bc98cdc86552411b P 4ef46d7a4eec4247881e5ae633b95839 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fcbcf1b2e12435999b48339cbfbd77c in term 1.
I20260807 10:50:47.644119 15235 raft_consensus.cc:2454] T dfaff5775d734ae7bc98cdc86552411b P cad93d47780840d69dae51ac99bd7bff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2fcbcf1b2e12435999b48339cbfbd77c in term 1.
I20260807 10:50:47.644569 14722 leader_election.cc:302] T dfaff5775d734ae7bc98cdc86552411b P 2fcbcf1b2e12435999b48339cbfbd77c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2fcbcf1b2e12435999b48339cbfbd77c, cad93d47780840d69dae51ac99bd7bff; no voters:
I20260807 10:50:47.644961 15424 raft_consensus.cc:2781] T dfaff5775d734ae7bc98cdc86552411b P 2fcbcf1b2e12435999b48339cbfbd77c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:50:47.645187 15424 raft_consensus.cc:686] T dfaff5775d734ae7bc98cdc86552411b P 2fcbcf1b2e12435999b48339cbfbd77c [term 1 LEADER]: Becoming Leader. State: Replica: 2fcbcf1b2e12435999b48339cbfbd77c, State: Running, Role: LEADER
I20260807 10:50:47.645488 15424 consensus_queue.cc:237] T dfaff5775d734ae7bc98cdc86552411b P 2fcbcf1b2e12435999b48339cbfbd77c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } }
I20260807 10:50:47.648088 14584 catalog_manager.cc:5495] T dfaff5775d734ae7bc98cdc86552411b P 2fcbcf1b2e12435999b48339cbfbd77c reported cstate change: term changed from 0 to 1, leader changed from <none> to 2fcbcf1b2e12435999b48339cbfbd77c (127.3.73.193). New cstate: current_term: 1 leader_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fcbcf1b2e12435999b48339cbfbd77c" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 } health_report { overall_health: UNKNOWN } } }
I20260807 10:50:47.682535 15463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:47.677162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37468 (local address 127.3.73.193:43623)
0807 10:50:47.677773 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:47.677787 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:47.677801 (+ 14us) server_negotiation.cc:407] Connection header received
0807 10:50:47.677859 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:47.677862 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:47.677933 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:47.678044 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:50:47.678452 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:47.679347 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:47.680751 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:47.681203 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:47.681362 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:47.681782 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:50:47.681890 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:50:47.681987 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:47.682143 (+ 156us) server_negotiation.cc:299] Negotiation successful
0807 10:50:47.682221 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":388,"threads_started":1}
I20260807 10:50:47.698086 15463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:47.691800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:33987 (local address 127.3.73.193:43623)
0807 10:50:47.691979 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:47.691985 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:47.692633 (+ 648us) server_negotiation.cc:407] Connection header received
0807 10:50:47.692836 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:47.692841 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:47.692918 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:47.693008 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:50:47.693492 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:47.694580 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:47.696125 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:47.696811 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:47.696926 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:47.697336 (+ 410us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:50:47.697457 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:50:47.697542 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:47.697686 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 10:50:47.697767 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 10:50:47.698088 15466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:47.691800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:43623 (local address 127.3.73.193:33987)
0807 10:50:47.692498 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:47.692516 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:47.692683 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:47.693076 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:47.693079 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:47.693090 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 10:50:47.693343 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:47.693352 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:47.694738 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:47.694744 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:47.695993 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:47.696007 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:47.696979 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:47.696983 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:47.697123 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:47.697160 (+ 37us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 10:50:47.697566 (+ 406us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 10:50:47.697576 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 10:50:47.697642 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 10:50:47.697743 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":371,"threads_started":1}
I20260807 10:50:47.707623 15235 raft_consensus.cc:1261] T dfaff5775d734ae7bc98cdc86552411b P cad93d47780840d69dae51ac99bd7bff [term 1 FOLLOWER]: Refusing update from remote peer 2fcbcf1b2e12435999b48339cbfbd77c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:50:47.707846 15022 raft_consensus.cc:1261] T dfaff5775d734ae7bc98cdc86552411b P 4ef46d7a4eec4247881e5ae633b95839 [term 1 FOLLOWER]: Refusing update from remote peer 2fcbcf1b2e12435999b48339cbfbd77c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:50:47.708379 15467 consensus_queue.cc:1035] T dfaff5775d734ae7bc98cdc86552411b P 2fcbcf1b2e12435999b48339cbfbd77c [LEADER]: Connected to new peer: Peer: permanent_uuid: "cad93d47780840d69dae51ac99bd7bff" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 41029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:50:47.708781 15424 consensus_queue.cc:1035] T dfaff5775d734ae7bc98cdc86552411b P 2fcbcf1b2e12435999b48339cbfbd77c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ef46d7a4eec4247881e5ae633b95839" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 46501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:50:47.760277 14584 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:50:47.760612 14584 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:50:47.763948 14584 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 014a586fde6e4cd8be60dd6be039aa30: Sending DeleteTablet for 3 replicas of tablet dfaff5775d734ae7bc98cdc86552411b
I20260807 10:50:47.764902 14792 tablet_service.cc:1514] Processing DeleteTablet for tablet dfaff5775d734ae7bc98cdc86552411b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:50:47 UTC) from {username='oryx', principal='oryx/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:37446
I20260807 10:50:47.765115 14997 tablet_service.cc:1514] Processing DeleteTablet for tablet dfaff5775d734ae7bc98cdc86552411b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:50:47 UTC) from {username='oryx', principal='oryx/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:57538
I20260807 10:50:47.765244 15384 tablet_replica.cc:330] T dfaff5775d734ae7bc98cdc86552411b P 2fcbcf1b2e12435999b48339cbfbd77c: stopping tablet replica
I20260807 10:50:47.765208 15200 tablet_service.cc:1514] Processing DeleteTablet for tablet dfaff5775d734ae7bc98cdc86552411b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:50:47 UTC) from {username='oryx', principal='oryx/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:39488
I20260807 10:50:47.765458 15384 raft_consensus.cc:2229] T dfaff5775d734ae7bc98cdc86552411b P 2fcbcf1b2e12435999b48339cbfbd77c [term 1 LEADER]: Raft consensus shutting down.
I20260807 10:50:47.765525 15383 tablet_replica.cc:330] T dfaff5775d734ae7bc98cdc86552411b P cad93d47780840d69dae51ac99bd7bff: stopping tablet replica
I20260807 10:50:47.765540 15386 tablet_replica.cc:330] T dfaff5775d734ae7bc98cdc86552411b P 4ef46d7a4eec4247881e5ae633b95839: stopping tablet replica
I20260807 10:50:47.765740 15386 raft_consensus.cc:2229] T dfaff5775d734ae7bc98cdc86552411b P 4ef46d7a4eec4247881e5ae633b95839 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:50:47.765750 15383 raft_consensus.cc:2229] T dfaff5775d734ae7bc98cdc86552411b P cad93d47780840d69dae51ac99bd7bff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:50:47.765945 15386 raft_consensus.cc:2258] T dfaff5775d734ae7bc98cdc86552411b P 4ef46d7a4eec4247881e5ae633b95839 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:50:47.765945 15383 raft_consensus.cc:2258] T dfaff5775d734ae7bc98cdc86552411b P cad93d47780840d69dae51ac99bd7bff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:50:47.766008 15384 raft_consensus.cc:2258] T dfaff5775d734ae7bc98cdc86552411b P 2fcbcf1b2e12435999b48339cbfbd77c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:50:47.767241 15384 ts_tablet_manager.cc:1905] T dfaff5775d734ae7bc98cdc86552411b P 2fcbcf1b2e12435999b48339cbfbd77c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:50:47.767253 15383 ts_tablet_manager.cc:1905] T dfaff5775d734ae7bc98cdc86552411b P cad93d47780840d69dae51ac99bd7bff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:50:47.768227 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 14692
I20260807 10:50:47.774868 15383 ts_tablet_manager.cc:1918] T dfaff5775d734ae7bc98cdc86552411b P cad93d47780840d69dae51ac99bd7bff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 10:50:47.775126 15383 log.cc:1196] T dfaff5775d734ae7bc98cdc86552411b P cad93d47780840d69dae51ac99bd7bff: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-2/wal/wals/dfaff5775d734ae7bc98cdc86552411b
I20260807 10:50:47.775646 15383 ts_tablet_manager.cc:1939] T dfaff5775d734ae7bc98cdc86552411b P cad93d47780840d69dae51ac99bd7bff: Deleting consensus metadata
I20260807 10:50:47.775759 15386 ts_tablet_manager.cc:1905] T dfaff5775d734ae7bc98cdc86552411b P 4ef46d7a4eec4247881e5ae633b95839: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:50:47.777024 14572 catalog_manager.cc:4841] TS cad93d47780840d69dae51ac99bd7bff (127.3.73.195:41029): tablet dfaff5775d734ae7bc98cdc86552411b (table test-table [id=40acc189baf44023892612549bb7da4c]) successfully deleted
I20260807 10:50:47.777275 15384 ts_tablet_manager.cc:1918] T dfaff5775d734ae7bc98cdc86552411b P 2fcbcf1b2e12435999b48339cbfbd77c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 10:50:47.777441 15384 log.cc:1196] T dfaff5775d734ae7bc98cdc86552411b P 2fcbcf1b2e12435999b48339cbfbd77c: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-0/wal/wals/dfaff5775d734ae7bc98cdc86552411b
I20260807 10:50:47.777886 15384 ts_tablet_manager.cc:1939] T dfaff5775d734ae7bc98cdc86552411b P 2fcbcf1b2e12435999b48339cbfbd77c: Deleting consensus metadata
I20260807 10:50:47.779130 14571 catalog_manager.cc:4841] TS 2fcbcf1b2e12435999b48339cbfbd77c (127.3.73.193:43623): tablet dfaff5775d734ae7bc98cdc86552411b (table test-table [id=40acc189baf44023892612549bb7da4c]) successfully deleted
I20260807 10:50:47.781708 15386 ts_tablet_manager.cc:1918] T dfaff5775d734ae7bc98cdc86552411b P 4ef46d7a4eec4247881e5ae633b95839: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 10:50:47.781870 15386 log.cc:1196] T dfaff5775d734ae7bc98cdc86552411b P 4ef46d7a4eec4247881e5ae633b95839: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786099767749326-3367-0/minicluster-data/ts-1/wal/wals/dfaff5775d734ae7bc98cdc86552411b
I20260807 10:50:47.782284 15386 ts_tablet_manager.cc:1939] T dfaff5775d734ae7bc98cdc86552411b P 4ef46d7a4eec4247881e5ae633b95839: Deleting consensus metadata
I20260807 10:50:47.783262 14571 catalog_manager.cc:4841] TS 4ef46d7a4eec4247881e5ae633b95839 (127.3.73.194:46501): tablet dfaff5775d734ae7bc98cdc86552411b (table test-table [id=40acc189baf44023892612549bb7da4c]) successfully deleted
Aug 07 10:50:47 dist-test-slave-x5fp krb5kdc[14524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.73.193@KRBTEST.COM
I20260807 10:50:47.803614 15463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:47.769465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37472 (local address 127.3.73.193:43623)
0807 10:50:47.769908 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:47.769917 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:47.787207 (+ 17290us) server_negotiation.cc:407] Connection header received
0807 10:50:47.787455 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:47.787461 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:47.787555 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:47.787674 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:47.789071 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:47.790257 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:47.791528 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:47.792209 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:47.792341 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:47.797064 (+ 4723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:47.797081 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:47.797099 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:47.797139 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:47.800525 (+ 3386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:47.801457 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:47.801465 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:47.801467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:47.801631 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:47.802046 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:47.802050 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:47.802052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:47.802429 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:47.802651 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:47.803046 (+ 395us) server_negotiation.cc:299] Negotiation successful
0807 10:50:47.803253 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286}
I20260807 10:50:47.804368 14891 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:47.944708 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 14692
I20260807 10:50:47.985570 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 14903
Aug 07 10:50:47 dist-test-slave-x5fp krb5kdc[14524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.73.194@KRBTEST.COM
I20260807 10:50:48.002226 15479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:47.986374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57550 (local address 127.3.73.194:46501)
0807 10:50:47.986960 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:47.986967 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:47.986987 (+ 20us) server_negotiation.cc:407] Connection header received
0807 10:50:47.987054 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:47.987057 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:47.987119 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:47.987256 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:47.988612 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:47.989570 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:47.990817 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:47.991259 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:47.991407 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:47.995877 (+ 4470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:47.995893 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:47.995910 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:47.995956 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:47.999146 (+ 3190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:47.999901 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:47.999909 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:47.999911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:48.000050 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:48.000464 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:48.000469 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:48.000472 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:48.000948 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:48.001295 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:48.001605 (+ 310us) server_negotiation.cc:299] Negotiation successful
0807 10:50:48.001843 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":300,"threads_started":1}
I20260807 10:50:48.002990 15099 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:48.105434 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 14903
I20260807 10:50:48.132462 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 15111
Aug 07 10:50:48 dist-test-slave-x5fp krb5kdc[14524](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.73.195@KRBTEST.COM
I20260807 10:50:48.149196 15482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:48.133337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39510 (local address 127.3.73.195:41029)
0807 10:50:48.133849 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:48.133862 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:48.133878 (+ 16us) server_negotiation.cc:407] Connection header received
0807 10:50:48.133933 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:48.133937 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:48.133998 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:48.134120 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:48.135516 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:48.136224 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:48.137698 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:48.138134 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:48.138297 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:48.143061 (+ 4764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:48.143079 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:48.143097 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:48.143143 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:48.146196 (+ 3053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:48.146950 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:48.146958 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:48.146961 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:48.147113 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:48.147523 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:48.147528 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:48.147531 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:48.147907 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:48.148203 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:48.148551 (+ 348us) server_negotiation.cc:299] Negotiation successful
0807 10:50:48.148827 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":253,"threads_started":1}
I20260807 10:50:48.150033 15301 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:48.279728 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 15111
I20260807 10:50:48.309796 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 14541
I20260807 10:50:48.311297 14647 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:48.433418 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 14541
2026-08-07T10:50:48Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6296 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:50:48 dist-test-slave-x5fp krb5kdc[15487](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:50:48 dist-test-slave-x5fp krb5kdc[15487](info): set up 2 sockets
Aug 07 10:50:48 dist-test-slave-x5fp krb5kdc[15487](info): commencing operation
krb5kdc: starting...
W20260807 10:50:49.866080 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.375s user 0.001s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:50:50 dist-test-slave-x5fp krb5kdc[15487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:50:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:50:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.73.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
I20260807 10:50:50.205099 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:44167
--webserver_interface=127.3.73.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:42829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:44167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:50:50.559252 15512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:50.559581 15512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:50.559757 15512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:50.567185 15512 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:50:50.567320 15512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:50.567413 15512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:50.567507 15512 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:50:50.567597 15512 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:50:50.577850 15512 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:42829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:44167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:44167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:50.579284 15512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:50.580780 15512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:50.593572 15519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:50.593523 15523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:50.593505 15520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:50.594604 15512 server_base.cc:1034] running on GCE node
I20260807 10:50:50.595566 15512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:50.597158 15512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:50.598419 15512 hybrid_clock.cc:648] HybridClock initialized: now 1786099850598366 us; error 44 us; skew 500 ppm
I20260807 10:50:50.598793 15512 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T10:50:50Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2227 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:50:50 dist-test-slave-x5fp krb5kdc[15535](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:50:50 dist-test-slave-x5fp krb5kdc[15535](info): set up 2 sockets
Aug 07 10:50:50 dist-test-slave-x5fp krb5kdc[15535](info): commencing operation
krb5kdc: starting...
W20260807 10:50:52.032615 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.315s user 0.003s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:50:52 dist-test-slave-x5fp krb5kdc[15535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:50:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:50:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.73.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
I20260807 10:50:52.354153 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:39487
--webserver_interface=127.3.73.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:39801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:39487
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:50:52.711066 15555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:52.711465 15555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:52.711642 15555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:52.719458 15555 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:50:52.719655 15555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:52.719764 15555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:52.719882 15555 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:50:52.719962 15555 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:50:52.730643 15555 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:39801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:39487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:39487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:52.732185 15555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:52.733661 15555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:52.746800 15561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:52.746938 15566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:52.746881 15562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:52.748080 15555 server_base.cc:1034] running on GCE node
I20260807 10:50:52.749166 15555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:52.750751 15555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:52.751979 15555 hybrid_clock.cc:648] HybridClock initialized: now 1786099852751930 us; error 34 us; skew 500 ppm
I20260807 10:50:52.752372 15555 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:52 dist-test-slave-x5fp krb5kdc[15535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099852, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:52.755844 15555 init.cc:377] Logged in from keytab as kudu/127.3.73.254@KRBTEST.COM (short username kudu)
I20260807 10:50:52.757570 15555 webserver.cc:460] Webserver started at http://127.3.73.254:42443/ using document root <none> and password file <none>
I20260807 10:50:52.758323 15555 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:52.758474 15555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:52.758802 15555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:52.761977 15555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "2a606bb74c2245e880b7120072c1ac4f"
format_stamp: "Formatted at 2026-08-07 10:50:52 on dist-test-slave-x5fp"
server_key: "06dccb9e3bcb03076c468c7d6cbd2f51"
server_key_iv: "5157dcd69040af76fdad3f910a63a428"
server_key_version: "encryptionkey@0"
I20260807 10:50:52.762797 15555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "2a606bb74c2245e880b7120072c1ac4f"
format_stamp: "Formatted at 2026-08-07 10:50:52 on dist-test-slave-x5fp"
server_key: "06dccb9e3bcb03076c468c7d6cbd2f51"
server_key_iv: "5157dcd69040af76fdad3f910a63a428"
server_key_version: "encryptionkey@0"
I20260807 10:50:52.768252 15555 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260807 10:50:52.771986 15574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:52.773656 15555 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 10:50:52.773867 15555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "2a606bb74c2245e880b7120072c1ac4f"
format_stamp: "Formatted at 2026-08-07 10:50:52 on dist-test-slave-x5fp"
server_key: "06dccb9e3bcb03076c468c7d6cbd2f51"
server_key_iv: "5157dcd69040af76fdad3f910a63a428"
server_key_version: "encryptionkey@0"
I20260807 10:50:52.774101 15555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:52.799063 15555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:52.803721 15555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:52.804044 15555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:52.824061 15555 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:39487
I20260807 10:50:52.824127 15641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:39487 every 8 connection(s)
I20260807 10:50:52.825538 15555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:50:52.826170 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 15555
I20260807 10:50:52.826449 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:50:52.827364 3367 external_mini_cluster.cc:1428] Setting key 2cf6e1b411e1292d466ca6574697057b
I20260807 10:50:52.830488 15642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 10:50:52 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:50:52.842338 15642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f: Bootstrap starting.
I20260807 10:50:52.846242 15642 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f: Neither blocks nor log segments found. Creating new log.
I20260807 10:50:52.847384 15642 log.cc:824] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f: Log is configured to *not* fsync() on all Append() calls
I20260807 10:50:52.850333 15648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:52.829577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44070 (local address 127.3.73.254:39487)
0807 10:50:52.830492 (+ 915us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:52.830504 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:52.830546 (+ 42us) server_negotiation.cc:407] Connection header received
0807 10:50:52.832335 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:52.832359 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:52.832822 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:52.833253 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:52.834877 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:52.836249 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:52.837584 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:52.838275 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:52.838471 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:52.842597 (+ 4126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:52.842606 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:52.842621 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:52.842658 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:52.845960 (+ 3302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:52.846746 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:52.846752 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:52.846758 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:52.846937 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:52.847344 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:52.847347 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:52.847350 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:52.848060 (+ 710us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:52.848417 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:52.848905 (+ 488us) server_negotiation.cc:299] Negotiation successful
0807 10:50:52.849348 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":694,"thread_start_us":381,"threads_started":1}
I20260807 10:50:52.850796 15642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f: No bootstrap required, opened a new log
I20260807 10:50:52.859808 15642 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a606bb74c2245e880b7120072c1ac4f" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 39487 } }
I20260807 10:50:52.860257 15642 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:50:52.860381 15642 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a606bb74c2245e880b7120072c1ac4f, State: Initialized, Role: FOLLOWER
I20260807 10:50:52.861053 15642 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a606bb74c2245e880b7120072c1ac4f" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 39487 } }
I20260807 10:50:52.861334 15642 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:50:52.861495 15642 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:50:52.861689 15642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:50:52.864544 15642 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a606bb74c2245e880b7120072c1ac4f" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 39487 } }
I20260807 10:50:52.865131 15642 leader_election.cc:302] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a606bb74c2245e880b7120072c1ac4f; no voters:
I20260807 10:50:52.865747 15642 leader_election.cc:288] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:50:52.865885 15651 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:50:52.866711 15651 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f [term 1 LEADER]: Becoming Leader. State: Replica: 2a606bb74c2245e880b7120072c1ac4f, State: Running, Role: LEADER
I20260807 10:50:52.867285 15651 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a606bb74c2245e880b7120072c1ac4f" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 39487 } }
I20260807 10:50:52.868119 15642 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:50:52.870541 15654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a606bb74c2245e880b7120072c1ac4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a606bb74c2245e880b7120072c1ac4f" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 39487 } } }
I20260807 10:50:52.871989 15654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:52.871233 15655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a606bb74c2245e880b7120072c1ac4f. Latest consensus state: current_term: 1 leader_uuid: "2a606bb74c2245e880b7120072c1ac4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a606bb74c2245e880b7120072c1ac4f" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 39487 } } }
I20260807 10:50:52.872828 15655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:52.873167 15659 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:50:52.877761 15659 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:50:52.891595 15659 catalog_manager.cc:1347] Generated new cluster ID: 256f7f0ab11249f78a09efe1d517651f
I20260807 10:50:52.891777 15659 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:50:52.909713 15659 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:50:52.911754 15659 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:50:52.938699 15659 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f: Generated new TSK 0
I20260807 10:50:52.939781 15659 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.73.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab.
Entry for principal kudu/127.3.73.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab.
I20260807 10:50:53.021895 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.193:0
--local_ip_for_outbound_sockets=127.3.73.193
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:39487
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:39801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:50:53.395320 15682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:53.395632 15682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:53.395793 15682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:53.403443 15682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:53.403576 15682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:53.403765 15682 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.193
I20260807 10:50:53.414386 15682 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:39801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:39487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.193
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:53.415854 15682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:53.417397 15682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:53.432209 15691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:53.432160 15692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:53.432262 15695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:53.438544 15682 server_base.cc:1034] running on GCE node
I20260807 10:50:53.439352 15682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:53.440579 15682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:53.441960 15682 hybrid_clock.cc:648] HybridClock initialized: now 1786099853441852 us; error 67 us; skew 500 ppm
I20260807 10:50:53.442353 15682 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:53 dist-test-slave-x5fp krb5kdc[15535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099853, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:53.446133 15682 init.cc:377] Logged in from keytab as kudu/127.3.73.193@KRBTEST.COM (short username kudu)
I20260807 10:50:53.448067 15682 webserver.cc:460] Webserver started at http://127.3.73.193:39419/ using document root <none> and password file <none>
I20260807 10:50:53.448964 15682 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:53.449127 15682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:53.449478 15682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:53.453001 15682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-0/data/instance:
uuid: "989e38625bab4aaf9692a75869c13564"
format_stamp: "Formatted at 2026-08-07 10:50:53 on dist-test-slave-x5fp"
server_key: "f66b7cdda46d1f2e1507caaa38b1d320"
server_key_iv: "a555dc6e465ab3e887965bf64a3413c2"
server_key_version: "encryptionkey@0"
I20260807 10:50:53.453984 15682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance:
uuid: "989e38625bab4aaf9692a75869c13564"
format_stamp: "Formatted at 2026-08-07 10:50:53 on dist-test-slave-x5fp"
server_key: "f66b7cdda46d1f2e1507caaa38b1d320"
server_key_iv: "a555dc6e465ab3e887965bf64a3413c2"
server_key_version: "encryptionkey@0"
I20260807 10:50:53.460178 15682 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 10:50:53.464636 15704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:53.466288 15682 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 10:50:53.466516 15682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-0/wal
uuid: "989e38625bab4aaf9692a75869c13564"
format_stamp: "Formatted at 2026-08-07 10:50:53 on dist-test-slave-x5fp"
server_key: "f66b7cdda46d1f2e1507caaa38b1d320"
server_key_iv: "a555dc6e465ab3e887965bf64a3413c2"
server_key_version: "encryptionkey@0"
I20260807 10:50:53.466754 15682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:53.490388 15682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:53.495471 15682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:53.495898 15682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:53.497100 15682 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:50:53.498698 15682 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:50:53.498826 15682 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:53.498993 15682 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:50:53.499073 15682 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:53.542721 15682 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.193:42481
I20260807 10:50:53.542754 15854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.193:42481 every 8 connection(s)
I20260807 10:50:53.544198 15682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
I20260807 10:50:53.546567 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 15682
I20260807 10:50:53.546828 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance
I20260807 10:50:53.547669 3367 external_mini_cluster.cc:1428] Setting key dc4156f78e4735043f2de080129bf90a
Aug 07 10:50:53 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099853, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:50:53.568849 15858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:53.547483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:46685 (local address 127.3.73.254:39487)
0807 10:50:53.548150 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:53.548157 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:53.549411 (+ 1254us) server_negotiation.cc:407] Connection header received
0807 10:50:53.551749 (+ 2338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:53.551756 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:53.551882 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:53.552029 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:53.554026 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:53.554898 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:53.556800 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:53.557288 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:53.557470 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:53.561870 (+ 4400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:53.561877 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:53.561880 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:53.561923 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:53.565395 (+ 3472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:53.566246 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:53.566251 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:53.566253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:53.566389 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:53.566892 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:53.566897 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:53.566899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:53.567192 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:53.567359 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:53.568196 (+ 837us) server_negotiation.cc:299] Negotiation successful
0807 10:50:53.568445 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":352,"threads_started":1}
I20260807 10:50:53.569929 15859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:53.548134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.193:46685)
0807 10:50:53.549201 (+ 1067us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:53.549257 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:53.551401 (+ 2144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:53.552330 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:53.552340 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:53.553065 (+ 725us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:53.553793 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:53.553824 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:53.555110 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:53.555115 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:53.556584 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:53.556599 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:53.557594 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:53.557605 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:53.557907 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:53.558771 (+ 864us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:53.558801 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:53.561524 (+ 2723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:53.565572 (+ 4048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:53.565584 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:53.565607 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:53.566088 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:53.566511 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:53.566516 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:53.566521 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:53.566726 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:53.567332 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:53.567340 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:53.567718 (+ 378us) client_negotiation.cc:769] Sending connection context
0807 10:50:53.568071 (+ 353us) client_negotiation.cc:240] Negotiation successful
0807 10:50:53.568696 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":736,"thread_start_us":444,"threads_started":1}
I20260807 10:50:53.571918 15855 heartbeater.cc:347] Connected to a master server at 127.3.73.254:39487
I20260807 10:50:53.572245 15855 heartbeater.cc:464] Registering TS with master...
I20260807 10:50:53.573076 15855 heartbeater.cc:511] Master 127.3.73.254:39487 requested a full tablet report, sending...
I20260807 10:50:53.575799 15598 ts_manager.cc:194] Registered new tserver with Master: 989e38625bab4aaf9692a75869c13564 (127.3.73.193:42481)
I20260807 10:50:53.578480 15598 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.193@KRBTEST.COM'} at 127.3.73.193:46685
WARNING: no policy specified for kudu/127.3.73.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab.
Entry for principal kudu/127.3.73.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab.
I20260807 10:50:53.624950 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.194:0
--local_ip_for_outbound_sockets=127.3.73.194
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:39487
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:39801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:50:53.988787 15864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:53.989197 15864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:53.989363 15864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:53.996976 15864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:53.997159 15864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:53.997347 15864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.194
I20260807 10:50:54.007697 15864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:39801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:39487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.194
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:54.009231 15864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:54.010633 15864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:54.025139 15873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:54.025175 15876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:54.025068 15871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:54.026137 15864 server_base.cc:1034] running on GCE node
I20260807 10:50:54.026964 15864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:54.028010 15864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:54.029263 15864 hybrid_clock.cc:648] HybridClock initialized: now 1786099854029200 us; error 54 us; skew 500 ppm
I20260807 10:50:54.029623 15864 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:54 dist-test-slave-x5fp krb5kdc[15535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099854, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:54.033241 15864 init.cc:377] Logged in from keytab as kudu/127.3.73.194@KRBTEST.COM (short username kudu)
I20260807 10:50:54.034989 15864 webserver.cc:460] Webserver started at http://127.3.73.194:33227/ using document root <none> and password file <none>
I20260807 10:50:54.035742 15864 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:54.035898 15864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:54.036252 15864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:54.039347 15864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-1/data/instance:
uuid: "fb947000e3d84e779879c37364fe059d"
format_stamp: "Formatted at 2026-08-07 10:50:54 on dist-test-slave-x5fp"
server_key: "b7327dec967476b50062d4ab071b4770"
server_key_iv: "71f592e95dc76317a0228a5d5cff5f96"
server_key_version: "encryptionkey@0"
I20260807 10:50:54.040171 15864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance:
uuid: "fb947000e3d84e779879c37364fe059d"
format_stamp: "Formatted at 2026-08-07 10:50:54 on dist-test-slave-x5fp"
server_key: "b7327dec967476b50062d4ab071b4770"
server_key_iv: "71f592e95dc76317a0228a5d5cff5f96"
server_key_version: "encryptionkey@0"
I20260807 10:50:54.045753 15864 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 10:50:54.049644 15887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:54.051186 15864 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 10:50:54.051410 15864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-1/wal
uuid: "fb947000e3d84e779879c37364fe059d"
format_stamp: "Formatted at 2026-08-07 10:50:54 on dist-test-slave-x5fp"
server_key: "b7327dec967476b50062d4ab071b4770"
server_key_iv: "71f592e95dc76317a0228a5d5cff5f96"
server_key_version: "encryptionkey@0"
I20260807 10:50:54.051651 15864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:54.082306 15864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:54.087507 15864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:54.087867 15864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:54.089046 15864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:50:54.090580 15864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:50:54.090695 15864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:54.090865 15864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:50:54.090945 15864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:54.128334 15864 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.194:35303
I20260807 10:50:54.128378 16053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.194:35303 every 8 connection(s)
I20260807 10:50:54.129676 15864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
I20260807 10:50:54.138556 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 15864
I20260807 10:50:54.138769 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance
I20260807 10:50:54.139250 3367 external_mini_cluster.cc:1428] Setting key 9d1857c6bc5e5c9f2a48fe812d316d5a
Aug 07 10:50:54 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099854, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:50:54.154079 16059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:54.132562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:45949 (local address 127.3.73.254:39487)
0807 10:50:54.133086 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:54.133090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:54.134272 (+ 1182us) server_negotiation.cc:407] Connection header received
0807 10:50:54.136409 (+ 2137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:54.136415 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:54.136525 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:54.136666 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:54.138354 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:54.139262 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:54.141657 (+ 2395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:54.142360 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:54.142516 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:54.146438 (+ 3922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:54.146447 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:54.146455 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:54.146502 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:54.150294 (+ 3792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:54.151089 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:54.151097 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:54.151103 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:54.151265 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:54.151761 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:54.151768 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:54.151773 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:54.152188 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:54.152350 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:54.153302 (+ 952us) server_negotiation.cc:299] Negotiation successful
0807 10:50:54.153563 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":285,"threads_started":1}
I20260807 10:50:54.154841 16060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:54.133254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.194:45949)
0807 10:50:54.134068 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:54.134116 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:54.136089 (+ 1973us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:54.136960 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:54.136969 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:54.137541 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:54.138150 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:54.138169 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:54.139424 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:54.139438 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:54.141435 (+ 1997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:54.141453 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:54.142650 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:54.142656 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:54.142867 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:54.143602 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:54.143627 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:54.146151 (+ 2524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:54.150458 (+ 4307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:54.150466 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:54.150479 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:54.150945 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:54.151414 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:54.151418 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:54.151420 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:54.151625 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:54.152355 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:54.152361 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:54.152849 (+ 488us) client_negotiation.cc:769] Sending connection context
0807 10:50:54.153146 (+ 297us) client_negotiation.cc:240] Negotiation successful
0807 10:50:54.153591 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":339,"threads_started":1}
I20260807 10:50:54.156599 16055 heartbeater.cc:347] Connected to a master server at 127.3.73.254:39487
I20260807 10:50:54.157039 16055 heartbeater.cc:464] Registering TS with master...
I20260807 10:50:54.157857 16055 heartbeater.cc:511] Master 127.3.73.254:39487 requested a full tablet report, sending...
I20260807 10:50:54.159518 15598 ts_manager.cc:194] Registered new tserver with Master: fb947000e3d84e779879c37364fe059d (127.3.73.194:35303)
I20260807 10:50:54.160432 15598 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.194@KRBTEST.COM'} at 127.3.73.194:45949
WARNING: no policy specified for kudu/127.3.73.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab.
Entry for principal kudu/127.3.73.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab.
I20260807 10:50:54.212486 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.195:0
--local_ip_for_outbound_sockets=127.3.73.195
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:39487
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:39801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:50:54.567788 16067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:54.568172 16067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:54.568356 16067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:54.576313 16067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:54.576493 16067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:54.576685 16067 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.195
I20260807 10:50:54.582742 15855 heartbeater.cc:503] Master 127.3.73.254:39487 was elected leader, sending a full tablet report...
I20260807 10:50:54.587634 16067 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:39801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:39487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.195
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:54.589128 16067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:54.590560 16067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:54.605406 16074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:54.605358 16078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:54.605305 16075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:54.606402 16067 server_base.cc:1034] running on GCE node
I20260807 10:50:54.607106 16067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:54.608124 16067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:54.609352 16067 hybrid_clock.cc:648] HybridClock initialized: now 1786099854609288 us; error 57 us; skew 500 ppm
I20260807 10:50:54.609680 16067 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:54 dist-test-slave-x5fp krb5kdc[15535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099854, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:54.613010 16067 init.cc:377] Logged in from keytab as kudu/127.3.73.195@KRBTEST.COM (short username kudu)
I20260807 10:50:54.614440 16067 webserver.cc:460] Webserver started at http://127.3.73.195:37593/ using document root <none> and password file <none>
I20260807 10:50:54.615118 16067 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:54.615216 16067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:54.615466 16067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:50:54.618415 16067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-2/data/instance:
uuid: "bf2c630c0291485b8f05ff19a5ed8b40"
format_stamp: "Formatted at 2026-08-07 10:50:54 on dist-test-slave-x5fp"
server_key: "c84f78deeee793d3fad3b82702797a1d"
server_key_iv: "c01a540aca7b7116043bdb6fcf6cb165"
server_key_version: "encryptionkey@0"
I20260807 10:50:54.619160 16067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance:
uuid: "bf2c630c0291485b8f05ff19a5ed8b40"
format_stamp: "Formatted at 2026-08-07 10:50:54 on dist-test-slave-x5fp"
server_key: "c84f78deeee793d3fad3b82702797a1d"
server_key_iv: "c01a540aca7b7116043bdb6fcf6cb165"
server_key_version: "encryptionkey@0"
I20260807 10:50:54.624655 16067 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 10:50:54.628315 16087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:54.629768 16067 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 10:50:54.629915 16067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-2/wal
uuid: "bf2c630c0291485b8f05ff19a5ed8b40"
format_stamp: "Formatted at 2026-08-07 10:50:54 on dist-test-slave-x5fp"
server_key: "c84f78deeee793d3fad3b82702797a1d"
server_key_iv: "c01a540aca7b7116043bdb6fcf6cb165"
server_key_version: "encryptionkey@0"
I20260807 10:50:54.630089 16067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:54.651404 16067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:54.656127 16067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:54.656405 16067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:54.657481 16067 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:50:54.658874 16067 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:50:54.658957 16067 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:54.659044 16067 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:50:54.659089 16067 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:54.694392 16067 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.195:34517
I20260807 10:50:54.694403 16269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.195:34517 every 8 connection(s)
I20260807 10:50:54.695705 16067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
I20260807 10:50:54.696110 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 16067
I20260807 10:50:54.696300 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance
I20260807 10:50:54.696990 3367 external_mini_cluster.cc:1428] Setting key e26552f4c4cdb9f9d0f9920d28535037
Aug 07 10:50:54 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099854, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:50:54.718076 16272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:54.698445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:44999 (local address 127.3.73.254:39487)
0807 10:50:54.698930 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:54.698938 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:54.700149 (+ 1211us) server_negotiation.cc:407] Connection header received
0807 10:50:54.702487 (+ 2338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:54.702493 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:54.702612 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:54.702766 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:54.704342 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:54.705261 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:54.706934 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:54.707360 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:54.707505 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:54.711175 (+ 3670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:54.711180 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:54.711183 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:54.711220 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:54.715127 (+ 3907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:54.715861 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:54.715866 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:54.715868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:54.715998 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:54.716389 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:54.716393 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:54.716395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:54.716672 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:54.716900 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:54.717550 (+ 650us) server_negotiation.cc:299] Negotiation successful
0807 10:50:54.717764 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":254,"threads_started":1}
I20260807 10:50:54.719023 16273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:54.698953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.195:44999)
0807 10:50:54.699959 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:54.700021 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:54.702206 (+ 2185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:54.703014 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:54.703023 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:54.703566 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:54.704148 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:54.704172 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:54.705418 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:54.705424 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:54.706763 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:54.706777 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:54.707612 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:54.707620 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:54.707876 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:54.708629 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:54.708652 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:54.710946 (+ 2294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:50:54.715265 (+ 4319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:54.715272 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:54.715286 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:54.715745 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:54.716099 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:50:54.716102 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:50:54.716104 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:50:54.716285 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:50:54.716853 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:50:54.716859 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:50:54.717194 (+ 335us) client_negotiation.cc:769] Sending connection context
0807 10:50:54.717438 (+ 244us) client_negotiation.cc:240] Negotiation successful
0807 10:50:54.717884 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":348,"threads_started":1}
I20260807 10:50:54.720444 16270 heartbeater.cc:347] Connected to a master server at 127.3.73.254:39487
I20260807 10:50:54.720806 16270 heartbeater.cc:464] Registering TS with master...
I20260807 10:50:54.721567 16270 heartbeater.cc:511] Master 127.3.73.254:39487 requested a full tablet report, sending...
I20260807 10:50:54.722992 15598 ts_manager.cc:194] Registered new tserver with Master: bf2c630c0291485b8f05ff19a5ed8b40 (127.3.73.195:34517)
I20260807 10:50:54.723827 15598 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.195@KRBTEST.COM'} at 127.3.73.195:44999
I20260807 10:50:54.726111 3367 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.3.73.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.73.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.73.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.193.keytab.
Entry for principal oryx/127.3.73.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.193.keytab.
WARNING: no policy specified for oryx/127.3.73.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.73.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.73.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.194.keytab.
Entry for principal oryx/127.3.73.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.194.keytab.
WARNING: no policy specified for oryx/127.3.73.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.73.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.73.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.195.keytab.
Entry for principal oryx/127.3.73.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.195.keytab.
I20260807 10:50:54.928388 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 15682
Aug 07 10:50:54 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.193@KRBTEST.COM
I20260807 10:50:54.946269 16287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:54.929759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52512 (local address 127.3.73.193:42481)
0807 10:50:54.930444 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:54.930452 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:54.930482 (+ 30us) server_negotiation.cc:407] Connection header received
0807 10:50:54.930547 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:54.930553 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:54.930770 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:54.930935 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:54.932364 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:54.933554 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:54.935163 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:54.935732 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:54.935836 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:54.940250 (+ 4414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:54.940263 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:54.940279 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:54.940317 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:54.943275 (+ 2958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:54.944054 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:54.944061 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:54.944063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:54.944192 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:54.944617 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:54.944620 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:54.944621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:54.945039 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:54.945270 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:54.945672 (+ 402us) server_negotiation.cc:299] Negotiation successful
0807 10:50:54.945978 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":351,"threads_started":1}
I20260807 10:50:54.947659 15848 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:55.033619 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 15682
I20260807 10:50:55.062975 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 15864
Aug 07 10:50:55 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.194@KRBTEST.COM
I20260807 10:50:55.086058 16289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:55.063948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53944 (local address 127.3.73.194:35303)
0807 10:50:55.064505 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:55.064513 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:55.064538 (+ 25us) server_negotiation.cc:407] Connection header received
0807 10:50:55.064601 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:55.064607 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:55.064893 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:55.065068 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:55.066409 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:55.067396 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:55.068926 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:55.069506 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:55.069654 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:55.079441 (+ 9787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:55.079458 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:55.079475 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:55.079537 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:55.082962 (+ 3425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:55.083723 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:55.083732 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:55.083734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:55.083878 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:55.084270 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:55.084276 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:55.084279 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:55.084659 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:55.085036 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:55.085352 (+ 316us) server_negotiation.cc:299] Negotiation successful
0807 10:50:55.085648 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":273,"threads_started":1}
I20260807 10:50:55.087512 16048 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:55.174011 16055 heartbeater.cc:503] Master 127.3.73.254:39487 was elected leader, sending a full tablet report...
I20260807 10:50:55.175177 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 15864
W20260807 10:50:55.199204 15576 connection.cc:535] server connection from 127.3.73.194:45949 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 10:50:55.199612 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 16067
Aug 07 10:50:55 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099852, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.195@KRBTEST.COM
I20260807 10:50:55.218547 16291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:55.200635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47384 (local address 127.3.73.195:34517)
0807 10:50:55.201268 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:55.201276 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:55.201308 (+ 32us) server_negotiation.cc:407] Connection header received
0807 10:50:55.201380 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:55.201386 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:55.201611 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:55.201762 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:55.203343 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:55.204383 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:55.206019 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:55.206627 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:55.206743 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:55.211546 (+ 4803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:55.211558 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:55.211573 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:55.211613 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:55.215416 (+ 3803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:55.216162 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:55.216169 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:55.216172 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:55.216326 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:55.216796 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:55.216799 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:55.216801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:55.217173 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:55.217432 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:55.217885 (+ 453us) server_negotiation.cc:299] Negotiation successful
0807 10:50:55.218133 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":315,"threads_started":1}
I20260807 10:50:55.220016 16258 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:55.304114 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 16067
I20260807 10:50:55.329326 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 15555
I20260807 10:50:55.330708 15636 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:50:55.439357 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 15555
I20260807 10:50:55.469709 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:39487
--webserver_interface=127.3.73.254
--webserver_port=42443
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:39801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:39487
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:50:55.841727 16298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:55.842046 16298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:55.842216 16298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:55.849906 16298 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:50:55.850068 16298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:55.850158 16298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:55.850256 16298 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:50:55.850348 16298 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:50:55.860914 16298 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:39801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:39487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:39487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=42443
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:55.862409 16298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:55.863878 16298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:55.876430 16309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:55.876541 16306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:55.876562 16307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:55.877643 16298 server_base.cc:1034] running on GCE node
I20260807 10:50:55.878504 16298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:55.879978 16298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:55.881224 16298 hybrid_clock.cc:648] HybridClock initialized: now 1786099855881174 us; error 45 us; skew 500 ppm
I20260807 10:50:55.881656 16298 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:55 dist-test-slave-x5fp krb5kdc[15535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099855, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:55.885329 16298 init.cc:377] Logged in from keytab as kudu/127.3.73.254@KRBTEST.COM (short username kudu)
I20260807 10:50:55.887192 16298 webserver.cc:460] Webserver started at http://127.3.73.254:42443/ using document root <none> and password file <none>
I20260807 10:50:55.887957 16298 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:55.888125 16298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:55.894284 16298 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260807 10:50:55.897589 16327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:55.899133 16298 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 10:50:55.899354 16298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "2a606bb74c2245e880b7120072c1ac4f"
format_stamp: "Formatted at 2026-08-07 10:50:52 on dist-test-slave-x5fp"
server_key: "06dccb9e3bcb03076c468c7d6cbd2f51"
server_key_iv: "5157dcd69040af76fdad3f910a63a428"
server_key_version: "encryptionkey@0"
I20260807 10:50:55.900491 16298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:55.927217 16298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:55.932271 16298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:55.932601 16298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:55.953603 16298 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:39487
I20260807 10:50:55.953639 16407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:39487 every 8 connection(s)
I20260807 10:50:55.955361 16298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:50:55.959106 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 16298
I20260807 10:50:55.960229 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.193:42481
--local_ip_for_outbound_sockets=127.3.73.193
--tserver_master_addrs=127.3.73.254:39487
--webserver_port=39419
--webserver_interface=127.3.73.193
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:39801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.3.73.193
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 10:50:55.965442 16411 sys_catalog.cc:263] Verifying existing consensus state
I20260807 10:50:55.968914 16411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f: Bootstrap starting.
I20260807 10:50:55.994272 16411 log.cc:824] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f: Log is configured to *not* fsync() on all Append() calls
I20260807 10:50:56.006907 16411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 10:50:56.007596 16411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f: Bootstrap complete.
I20260807 10:50:56.021675 16411 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a606bb74c2245e880b7120072c1ac4f" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 39487 } }
I20260807 10:50:56.022904 16411 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a606bb74c2245e880b7120072c1ac4f, State: Initialized, Role: FOLLOWER
I20260807 10:50:56.023672 16411 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a606bb74c2245e880b7120072c1ac4f" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 39487 } }
I20260807 10:50:56.023974 16411 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:50:56.024098 16411 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:50:56.024277 16411 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f [term 1 FOLLOWER]: Advancing to term 2
I20260807 10:50:56.030273 16411 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a606bb74c2245e880b7120072c1ac4f" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 39487 } }
I20260807 10:50:56.030985 16411 leader_election.cc:302] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a606bb74c2245e880b7120072c1ac4f; no voters:
I20260807 10:50:56.031699 16411 leader_election.cc:288] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 10:50:56.032012 16419 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f [term 2 FOLLOWER]: Leader election won for term 2
I20260807 10:50:56.033589 16419 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f [term 2 LEADER]: Becoming Leader. State: Replica: 2a606bb74c2245e880b7120072c1ac4f, State: Running, Role: LEADER
I20260807 10:50:56.034344 16411 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:50:56.034253 16419 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a606bb74c2245e880b7120072c1ac4f" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 39487 } }
I20260807 10:50:56.037068 16423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a606bb74c2245e880b7120072c1ac4f. Latest consensus state: current_term: 2 leader_uuid: "2a606bb74c2245e880b7120072c1ac4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a606bb74c2245e880b7120072c1ac4f" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 39487 } } }
I20260807 10:50:56.037520 16423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:56.038105 16422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2a606bb74c2245e880b7120072c1ac4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a606bb74c2245e880b7120072c1ac4f" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 39487 } } }
I20260807 10:50:56.038491 16422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f [sys.catalog]: This master's current role is: LEADER
I20260807 10:50:56.048655 16431 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:50:56.055547 16431 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:50:56.057214 16431 catalog_manager.cc:1259] Loaded cluster ID: 256f7f0ab11249f78a09efe1d517651f
I20260807 10:50:56.057389 16431 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:50:56.062068 16431 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:50:56.063756 16431 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2a606bb74c2245e880b7120072c1ac4f: Loaded TSK: 0
I20260807 10:50:56.065461 16431 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 10:50:56.351397 16414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:56.351749 16414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:56.351918 16414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:56.359611 16414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:56.359791 16414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:56.359982 16414 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.193
I20260807 10:50:56.370358 16414 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:39801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.193.keytab
--principal=oryx/127.3.73.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.193:42481
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.73.193
--webserver_port=39419
--tserver_master_addrs=127.3.73.254:39487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.193
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:56.371850 16414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:56.373406 16414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:56.390858 16454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:56.391449 16455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:56.391911 16457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:56.393332 16414 server_base.cc:1034] running on GCE node
I20260807 10:50:56.394177 16414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:56.395411 16414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:56.396768 16414 hybrid_clock.cc:648] HybridClock initialized: now 1786099856396688 us; error 37 us; skew 500 ppm
I20260807 10:50:56.397262 16414 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:56 dist-test-slave-x5fp krb5kdc[15535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099856, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:56.401315 16414 init.cc:377] Logged in from keytab as oryx/127.3.73.193@KRBTEST.COM (short username oryx)
I20260807 10:50:56.403142 16414 webserver.cc:460] Webserver started at http://127.3.73.193:39419/ using document root <none> and password file <none>
I20260807 10:50:56.404091 16414 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:56.404289 16414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:56.411104 16414 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 10:50:56.414417 16467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:56.416033 16414 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 10:50:56.416246 16414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-0/wal
uuid: "989e38625bab4aaf9692a75869c13564"
format_stamp: "Formatted at 2026-08-07 10:50:53 on dist-test-slave-x5fp"
server_key: "f66b7cdda46d1f2e1507caaa38b1d320"
server_key_iv: "a555dc6e465ab3e887965bf64a3413c2"
server_key_version: "encryptionkey@0"
I20260807 10:50:56.417435 16414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:56.440587 16414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:56.445953 16414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:56.446285 16414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:56.447413 16414 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:50:56.448980 16414 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:50:56.449106 16414 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:56.449262 16414 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:50:56.449358 16414 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:56.485941 16414 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.193:42481
I20260807 10:50:56.485968 16630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.193:42481 every 8 connection(s)
I20260807 10:50:56.487524 16414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
I20260807 10:50:56.490403 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 16414
I20260807 10:50:56.491286 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.194:35303
--local_ip_for_outbound_sockets=127.3.73.194
--tserver_master_addrs=127.3.73.254:39487
--webserver_port=33227
--webserver_interface=127.3.73.194
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:39801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.3.73.194
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 10:50:56 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:50:56 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 10:50:56.508528 16636 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:50:56.491103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:46099 (local address 127.3.73.254:39487)
0807 10:50:56.493302 (+ 2199us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:56.493318 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:56.493377 (+ 59us) server_negotiation.cc:407] Connection header received
0807 10:50:56.494976 (+ 1599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:56.494990 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:56.495337 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:56.495702 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:56.497400 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:56.498687 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:56.500293 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:56.501028 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:56.501257 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:56.507546 (+ 6289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.193:46099: BlockingRecv error: recv got EOF from 127.3.73.193:46099 (error 108)
Metrics: {"server-negotiator.queue_time_us":1913,"thread_start_us":1670,"threads_started":1}
I20260807 10:50:56.508601 16635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:56.491018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.193:46099)
0807 10:50:56.492212 (+ 1194us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:56.492262 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:56.494360 (+ 2098us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:56.495878 (+ 1518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:56.495888 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:56.496528 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:56.497160 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:56.497179 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:56.498875 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:56.498879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:56.500127 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:56.500140 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:56.501299 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:56.501307 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:56.501617 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:56.502352 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:56.502382 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:56.507226 (+ 4844us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":881,"thread_start_us":674,"threads_started":1}
W20260807 10:50:56.509481 16631 heartbeater.cc:644] Failed to heartbeat to 127.3.73.254:39487 (0 consecutive failures): Not authorized: Failed to ping master at 127.3.73.254:39487: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Aug 07 10:50:56 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:50:56 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:50:56.523197 16635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:56.510222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.193:42455)
0807 10:50:56.510424 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:56.510441 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:56.510875 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:56.511217 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:56.511221 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:56.511606 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:56.511745 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:56.511754 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:56.512867 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:56.512874 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:56.514026 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:56.514072 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:56.515139 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:56.515146 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:56.515450 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:56.516915 (+ 1465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:56.516973 (+ 58us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:56.522866 (+ 5893us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 10:50:56.523581 16636 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:50:56.510421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:42455 (local address 127.3.73.254:39487)
0807 10:50:56.510894 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:56.510899 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:56.510920 (+ 21us) server_negotiation.cc:407] Connection header received
0807 10:50:56.511008 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:56.511013 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:56.511082 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:56.511187 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:56.511896 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:56.512670 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:56.514378 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:56.514966 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:56.515125 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:56.523114 (+ 7989us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.193:42455: BlockingRecv error: recv got EOF from 127.3.73.193:42455 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
Aug 07 10:50:56 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:50:56 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:50:56.556252 16635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:56.525013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.193:37939)
0807 10:50:56.525453 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:56.525514 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:56.525733 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:56.527019 (+ 1286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:56.527039 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:56.527533 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:56.527801 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:56.527813 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:56.532468 (+ 4655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:56.532478 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:56.537764 (+ 5286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:56.537861 (+ 97us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:56.540412 (+ 2551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:56.540422 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:56.541661 (+ 1239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:56.543996 (+ 2335us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:56.544266 (+ 270us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:56.555881 (+ 11615us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":75}
W20260807 10:50:56.556450 16636 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:50:56.525132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:37939 (local address 127.3.73.254:39487)
0807 10:50:56.525824 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:56.525837 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:56.525976 (+ 139us) server_negotiation.cc:407] Connection header received
0807 10:50:56.526278 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:56.526288 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:56.526575 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:56.526948 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:56.528140 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:56.532110 (+ 3970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:56.538669 (+ 6559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:56.539995 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:56.540175 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:56.556187 (+ 16012us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.193:37939: BlockingRecv error: recv got EOF from 127.3.73.193:37939 (error 108)
Metrics: {"server-negotiator.queue_time_us":311}
W20260807 10:50:56.556690 16631 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 10:50:57.023501 16637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:57.023909 16637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:57.024103 16637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:57.032303 16637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:57.032529 16637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:57.032748 16637 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.194
I20260807 10:50:57.043659 16637 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:39801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.194.keytab
--principal=oryx/127.3.73.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.194:35303
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.73.194
--webserver_port=33227
--tserver_master_addrs=127.3.73.254:39487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.194
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:57.045222 16637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:57.046711 16637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:57.062165 16642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:57.062114 16643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:57.062732 16645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:57.063700 16637 server_base.cc:1034] running on GCE node
I20260807 10:50:57.064518 16637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:57.065721 16637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:57.067051 16637 hybrid_clock.cc:648] HybridClock initialized: now 1786099857066991 us; error 55 us; skew 500 ppm
I20260807 10:50:57.067480 16637 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:57 dist-test-slave-x5fp krb5kdc[15535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099857, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:57.071089 16637 init.cc:377] Logged in from keytab as oryx/127.3.73.194@KRBTEST.COM (short username oryx)
I20260807 10:50:57.072993 16637 webserver.cc:460] Webserver started at http://127.3.73.194:33227/ using document root <none> and password file <none>
I20260807 10:50:57.073783 16637 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:57.073964 16637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:57.080193 16637 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260807 10:50:57.083504 16652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:57.085328 16637 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260807 10:50:57.085594 16637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-1/wal
uuid: "fb947000e3d84e779879c37364fe059d"
format_stamp: "Formatted at 2026-08-07 10:50:54 on dist-test-slave-x5fp"
server_key: "b7327dec967476b50062d4ab071b4770"
server_key_iv: "71f592e95dc76317a0228a5d5cff5f96"
server_key_version: "encryptionkey@0"
I20260807 10:50:57.086826 16637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:57.108536 16637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:57.114022 16637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:57.114362 16637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:57.115574 16637 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:50:57.117403 16637 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:50:57.117553 16637 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:57.117707 16637 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:50:57.117807 16637 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:57.158790 16637 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.194:35303
I20260807 10:50:57.158924 16826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.194:35303 every 8 connection(s)
I20260807 10:50:57.160493 16637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
I20260807 10:50:57.162415 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 16637
I20260807 10:50:57.163170 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.195:34517
--local_ip_for_outbound_sockets=127.3.73.195
--tserver_master_addrs=127.3.73.254:39487
--webserver_port=37593
--webserver_interface=127.3.73.195
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:39801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.3.73.195
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 10:50:57 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:50:57 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 10:50:57.179085 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:50:57.163917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:38467 (local address 127.3.73.254:39487)
0807 10:50:57.165163 (+ 1246us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:57.165168 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:57.165886 (+ 718us) server_negotiation.cc:407] Connection header received
0807 10:50:57.167906 (+ 2020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:57.167912 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:57.168018 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:57.168144 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:57.169869 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:57.170651 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:57.172300 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:57.172849 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:57.172996 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:57.178771 (+ 5775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.194:38467: BlockingRecv error: recv got EOF from 127.3.73.194:38467 (error 108)
Metrics: {"server-negotiator.queue_time_us":1125,"thread_start_us":1002,"threads_started":1}
I20260807 10:50:57.179829 16832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:57.164592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.194:38467)
0807 10:50:57.165695 (+ 1103us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:57.165745 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:57.167596 (+ 1851us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:57.168430 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:57.168440 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:57.169073 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:57.169669 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:57.169687 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:57.170855 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:57.170859 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:57.172136 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:57.172149 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:57.173108 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:57.173117 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:57.173414 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:57.174201 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:57.174228 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:57.178528 (+ 4300us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":835,"thread_start_us":590,"threads_started":1}
W20260807 10:50:57.180651 16828 heartbeater.cc:644] Failed to heartbeat to 127.3.73.254:39487 (0 consecutive failures): Not authorized: Failed to ping master at 127.3.73.254:39487: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Aug 07 10:50:57 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:50:57 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:50:57.192222 16832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:57.181462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.194:52943)
0807 10:50:57.181714 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:57.181781 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:57.181916 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:57.182304 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:57.182308 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:57.182682 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:57.182922 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:57.182933 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:57.184621 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:57.184626 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:57.185867 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:57.185882 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:57.187142 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:57.187148 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:57.187285 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:57.188084 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:57.188108 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:57.191692 (+ 3584us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":75}
W20260807 10:50:57.192355 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:50:57.181629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:52943 (local address 127.3.73.254:39487)
0807 10:50:57.181869 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:57.181879 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:57.181907 (+ 28us) server_negotiation.cc:407] Connection header received
0807 10:50:57.182034 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:57.182043 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:57.182140 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:57.182263 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:57.183108 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:57.184452 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:57.186038 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:57.186947 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:57.187111 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:57.192020 (+ 4909us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.194:52943: BlockingRecv error: recv got EOF from 127.3.73.194:52943 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 07 10:50:57 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:50:57 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:50:57.204640 16832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:57.192992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.194:34501)
0807 10:50:57.193241 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:57.193267 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:57.193414 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:57.194059 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:57.194081 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:57.194617 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:57.194892 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:57.194909 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:57.196066 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:57.196074 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:57.198174 (+ 2100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:57.198199 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:57.198992 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:57.199000 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:57.199300 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:57.200111 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:57.200144 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:57.204223 (+ 4079us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":693,"threads_started":1}
W20260807 10:50:57.204977 16828 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 10:50:57.205319 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:50:57.193137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:34501 (local address 127.3.73.254:39487)
0807 10:50:57.193360 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:57.193369 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:57.193408 (+ 39us) server_negotiation.cc:407] Connection header received
0807 10:50:57.193586 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:57.193594 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:57.193700 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:57.193828 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:57.195096 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:57.195887 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:57.198355 (+ 2468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:57.198803 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:57.198920 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:57.204498 (+ 5578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.194:34501: BlockingRecv error: recv got EOF from 127.3.73.194:34501 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260807 10:50:57.554530 16833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:50:57.554930 16833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:50:57.555114 16833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:50:57.565757 16833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:50:57.566085 16833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:50:57.566362 16833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.195
Aug 07 10:50:57 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:50:57 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:50:57.574543 16841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:57.557562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.193:45145)
0807 10:50:57.558142 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:57.558159 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:57.558355 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:57.558833 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:57.558842 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:57.559900 (+ 1058us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:57.560487 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:57.560508 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:57.562557 (+ 2049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:57.562564 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:57.565686 (+ 3122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:57.565723 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:57.567806 (+ 2083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:57.567814 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:57.568065 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:57.569372 (+ 1307us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:57.569400 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:57.574168 (+ 4768us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":295,"threads_started":1}
W20260807 10:50:57.575271 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:50:57.557708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:45145 (local address 127.3.73.254:39487)
0807 10:50:57.557995 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:57.558002 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:57.558311 (+ 309us) server_negotiation.cc:407] Connection header received
0807 10:50:57.558524 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:57.558531 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:57.558628 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:57.558746 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:57.560772 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:57.562244 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:57.566048 (+ 3804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:57.567524 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:57.567751 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:57.574605 (+ 6854us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.193:45145: BlockingRecv error: recv got EOF from 127.3.73.193:45145 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
I20260807 10:50:57.582614 16833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:39801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/krb5kdc/oryx_127.3.73.195.keytab
--principal=oryx/127.3.73.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.195:34517
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.73.195
--webserver_port=37593
--tserver_master_addrs=127.3.73.254:39487
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.195
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:50:57.584398 16833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:50:57.586036 16833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:50:57.607316 16844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:57.607271 16845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:50:57.607815 16848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:50:57.609359 16833 server_base.cc:1034] running on GCE node
I20260807 10:50:57.610158 16833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:50:57.611368 16833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:50:57.612776 16833 hybrid_clock.cc:648] HybridClock initialized: now 1786099857612570 us; error 65 us; skew 500 ppm
I20260807 10:50:57.613271 16833 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:50:57 dist-test-slave-x5fp krb5kdc[15535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099857, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.73.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:50:57.618117 16833 init.cc:377] Logged in from keytab as oryx/127.3.73.195@KRBTEST.COM (short username oryx)
I20260807 10:50:57.620174 16833 webserver.cc:460] Webserver started at http://127.3.73.195:37593/ using document root <none> and password file <none>
I20260807 10:50:57.621255 16833 fs_manager.cc:359] Metadata directory not provided
I20260807 10:50:57.621513 16833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:50:57.628696 16833 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 10:50:57.632676 16858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:57.634598 16833 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260807 10:50:57.634859 16833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-2/wal
uuid: "bf2c630c0291485b8f05ff19a5ed8b40"
format_stamp: "Formatted at 2026-08-07 10:50:54 on dist-test-slave-x5fp"
server_key: "c84f78deeee793d3fad3b82702797a1d"
server_key_iv: "c01a540aca7b7116043bdb6fcf6cb165"
server_key_version: "encryptionkey@0"
I20260807 10:50:57.636407 16833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:50:57.659013 16833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:50:57.665259 16833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:50:57.665721 16833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:50:57.667281 16833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:50:57.668817 16833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:50:57.668934 16833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:57.669067 16833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:50:57.669147 16833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:50:57.710247 16833 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.195:34517
I20260807 10:50:57.710346 17022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.195:34517 every 8 connection(s)
I20260807 10:50:57.711934 16833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
I20260807 10:50:57.713912 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 16833
Aug 07 10:50:57 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:50:57 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 10:50:57.731552 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:50:57.715024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:50379 (local address 127.3.73.254:39487)
0807 10:50:57.715259 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:57.715267 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:57.717261 (+ 1994us) server_negotiation.cc:407] Connection header received
0807 10:50:57.719437 (+ 2176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:57.719445 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:57.719555 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:57.719672 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:57.721514 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:57.722417 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:57.724422 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:57.724889 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:57.725029 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:57.731005 (+ 5976us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.195:50379: BlockingRecv error: recv got EOF from 127.3.73.195:50379 (error 108)
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":90}
I20260807 10:50:57.732167 17029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:57.715560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.195:50379)
0807 10:50:57.717104 (+ 1544us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:57.717147 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:57.719080 (+ 1933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:57.719949 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:57.719962 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:57.720504 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:57.721314 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:57.721335 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:57.722595 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:57.722599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:57.724264 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:57.724277 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:57.725293 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:57.725305 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:57.725652 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:57.726600 (+ 948us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:57.726656 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:57.730709 (+ 4053us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1284,"thread_start_us":402,"threads_started":1}
W20260807 10:50:57.733214 17023 heartbeater.cc:644] Failed to heartbeat to 127.3.73.254:39487 (0 consecutive failures): Not authorized: Failed to ping master at 127.3.73.254:39487: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
I20260807 10:50:57.736318 17028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:57.715024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44082 (local address 127.3.73.254:39487)
0807 10:50:57.715812 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:57.715820 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:57.715850 (+ 30us) server_negotiation.cc:407] Connection header received
0807 10:50:57.715923 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:57.715932 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:57.716051 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:57.716189 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:57.718705 (+ 2516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:57.719870 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:57.721684 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:57.722423 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:57.722556 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:57.726942 (+ 4386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:50:57.726952 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:50:57.726969 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:50:57.727007 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:50:57.730648 (+ 3641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:57.731574 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:57.731583 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:57.731597 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:57.732064 (+ 467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:50:57.732586 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:50:57.732592 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:50:57.732595 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:50:57.734148 (+ 1553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:50:57.734536 (+ 388us) server_negotiation.cc:1035] Waiting for connection context
0807 10:50:57.735050 (+ 514us) server_negotiation.cc:299] Negotiation successful
0807 10:50:57.735597 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":292,"server-negotiator.queue_time_us":639,"thread_start_us":259,"threads_started":1}
Aug 07 10:50:57 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:50:57 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:50:57.744448 17029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:57.733881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.195:52517)
0807 10:50:57.734064 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:57.734084 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:57.734197 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:57.734722 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:57.734728 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:57.735261 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:57.735453 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:57.735462 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:57.737030 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:57.737036 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:57.738632 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:57.738646 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:57.739753 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:57.739757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:57.739897 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:57.740802 (+ 905us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:57.740825 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:57.744069 (+ 3244us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 10:50:57.744761 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:50:57.733970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:52517 (local address 127.3.73.254:39487)
0807 10:50:57.734275 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:57.734282 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:57.734300 (+ 18us) server_negotiation.cc:407] Connection header received
0807 10:50:57.734345 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:57.734355 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:57.734408 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:57.734520 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:57.735597 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:57.736840 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:57.738781 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:57.739459 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:57.739583 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:57.744351 (+ 4768us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.195:52517: BlockingRecv error: recv got EOF from 127.3.73.195:52517 (error 108)
Metrics: {"server-negotiator.queue_time_us":184}
Aug 07 10:50:57 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:50:57 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:50:57.756039 17029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:57.745479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.195:47015)
0807 10:50:57.745689 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:57.745719 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:57.745840 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:57.746382 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:57.746389 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:57.746837 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:57.747072 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:57.747086 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:57.748443 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:57.748451 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:57.750338 (+ 1887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:57.750356 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:57.751373 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:57.751377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:57.751608 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:57.752394 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:57.752412 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:57.755769 (+ 3357us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":60}
W20260807 10:50:57.756304 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:50:57.745567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:47015 (local address 127.3.73.254:39487)
0807 10:50:57.745721 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:57.745725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:57.745786 (+ 61us) server_negotiation.cc:407] Connection header received
0807 10:50:57.745990 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:57.745998 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:57.746079 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:57.746150 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:57.747223 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:57.748269 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:57.750542 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:57.751233 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:57.751331 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:57.756040 (+ 4709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.195:47015: BlockingRecv error: recv got EOF from 127.3.73.195:47015 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260807 10:50:57.756551 17023 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 10:50:58 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:50:58 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:50:58.217497 17030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:58.205823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.194:36815)
0807 10:50:58.206489 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:58.206506 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:58.206640 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:58.207085 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:58.207089 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:58.207722 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:58.208070 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:58.208083 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:58.209512 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:58.209516 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:58.210736 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:58.210751 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:58.211746 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:58.211751 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:58.211951 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:58.212898 (+ 947us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:58.212923 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:58.217232 (+ 4309us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":406,"threads_started":1}
W20260807 10:50:58.217736 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:50:58.205824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:36815 (local address 127.3.73.254:39487)
0807 10:50:58.206154 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:58.206164 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:58.206604 (+ 440us) server_negotiation.cc:407] Connection header received
0807 10:50:58.206761 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:58.206765 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:58.206842 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:58.206947 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:58.208244 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:58.209336 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:58.210909 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:58.211528 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:58.211634 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:58.217500 (+ 5866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.194:36815: BlockingRecv error: recv got EOF from 127.3.73.194:36815 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 07 10:50:58 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:50:58 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:50:58.585696 17031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:58.575858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.193:60265)
0807 10:50:58.576622 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:58.576642 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:58.576821 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:58.577279 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:58.577285 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:58.577721 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:58.577928 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:58.577938 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:58.579087 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:58.579091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:58.580063 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:58.580076 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:58.580989 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:58.580999 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:58.581172 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:58.582049 (+ 877us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:58.582069 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:58.585316 (+ 3247us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":442,"threads_started":1}
W20260807 10:50:58.585801 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:50:58.575929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:60265 (local address 127.3.73.254:39487)
0807 10:50:58.576171 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:58.576176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:58.576786 (+ 610us) server_negotiation.cc:407] Connection header received
0807 10:50:58.576978 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:58.576983 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:58.577073 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:58.577176 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:58.578124 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:58.578910 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:58.580214 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:58.580758 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:58.580918 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:58.585533 (+ 4615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.193:60265: BlockingRecv error: recv got EOF from 127.3.73.193:60265 (error 108)
Metrics: {"server-negotiator.queue_time_us":116}
Aug 07 10:50:58 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:50:58 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:50:58.767117 17037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:58.757323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.195:52651)
0807 10:50:58.758044 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:58.758073 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:58.758245 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:58.758675 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:58.758680 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:58.759377 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:58.759657 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:58.759672 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:58.760775 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:58.760779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:58.761803 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:58.761816 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:58.762557 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:58.762561 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:58.762680 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:58.763281 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:58.763298 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:58.766724 (+ 3426us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":340,"threads_started":1}
W20260807 10:50:58.767186 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:50:58.757591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:52651 (local address 127.3.73.254:39487)
0807 10:50:58.757824 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:58.757831 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:58.758171 (+ 340us) server_negotiation.cc:407] Connection header received
0807 10:50:58.758393 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:58.758402 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:58.758466 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:58.758589 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:58.759808 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:58.760605 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:58.761949 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:58.762387 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:58.762486 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:58.766943 (+ 4457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.195:52651: BlockingRecv error: recv got EOF from 127.3.73.195:52651 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 07 10:50:59 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:50:59 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:50:59.228632 17038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:59.218579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.194:43289)
0807 10:50:59.219159 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:59.219176 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:59.219359 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:59.219731 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:59.219736 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:59.220234 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:59.220411 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:59.220421 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:59.221736 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:59.221741 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:59.222787 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:59.222801 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:59.223536 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:59.223542 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:59.223718 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:59.224445 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:59.224473 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:59.228267 (+ 3794us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":324,"threads_started":1}
W20260807 10:50:59.228746 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:50:59.218723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:43289 (local address 127.3.73.254:39487)
0807 10:50:59.218969 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:59.218977 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:59.219282 (+ 305us) server_negotiation.cc:407] Connection header received
0807 10:50:59.219498 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:59.219504 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:59.219577 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:59.219687 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:59.220567 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:59.221549 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:59.222934 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:59.223369 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:59.223514 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:59.228473 (+ 4959us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.194:43289: BlockingRecv error: recv got EOF from 127.3.73.194:43289 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 07 10:50:59 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:50:59 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:50:59.596860 17039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:59.586832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.193:44535)
0807 10:50:59.587420 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:59.587437 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:59.587564 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:59.588005 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:59.588009 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:59.588363 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:59.588573 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:59.588583 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:59.589922 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:59.589933 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:59.591273 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:59.591293 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:59.592062 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:59.592068 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:59.592291 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:59.593142 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:59.593163 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:59.596476 (+ 3313us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":319,"threads_started":1}
W20260807 10:50:59.597115 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:50:59.587144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:44535 (local address 127.3.73.254:39487)
0807 10:50:59.587359 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:59.587364 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:59.587523 (+ 159us) server_negotiation.cc:407] Connection header received
0807 10:50:59.587748 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:59.587753 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:59.587847 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:59.587965 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:59.588822 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:59.589737 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:59.591444 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:59.591899 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:59.592043 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:59.596834 (+ 4791us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.193:44535: BlockingRecv error: recv got EOF from 127.3.73.193:44535 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 07 10:50:59 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:50:59 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:50:59.777098 17041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:50:59.768400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.195:59841)
0807 10:50:59.769140 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0807 10:50:59.769158 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 10:50:59.769334 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:50:59.769707 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:50:59.769711 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:50:59.770148 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:50:59.770295 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:59.770305 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:59.771360 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:59.771365 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:59.772264 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:50:59.772275 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:50:59.773068 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:50:59.773074 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:50:59.773244 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:59.773850 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:50:59.773868 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:50:59.776707 (+ 2839us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":491,"threads_started":1}
W20260807 10:50:59.777246 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:50:59.768531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:59841 (local address 127.3.73.254:39487)
0807 10:50:59.768763 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 10:50:59.768769 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:50:59.769260 (+ 491us) server_negotiation.cc:407] Connection header received
0807 10:50:59.769475 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:50:59.769486 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:50:59.769552 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:50:59.769667 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:50:59.770451 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:59.771197 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:59.772410 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:50:59.772905 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:50:59.773044 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:50:59.776977 (+ 3933us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.195:59841: BlockingRecv error: recv got EOF from 127.3.73.195:59841 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 07 10:51:00 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:00 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:00.247902 17045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:00.230091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.194:58785)
0807 10:51:00.230775 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:00.230796 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:00.230942 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:00.231424 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:00.231428 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:00.231824 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:00.232061 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:00.232074 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:00.233451 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:00.233456 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:00.234660 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:00.234676 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:00.235557 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:00.235562 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:00.235848 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:00.243952 (+ 8104us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:00.243977 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:00.247573 (+ 3596us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":390,"threads_started":1}
W20260807 10:51:00.248247 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:00.230179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:58785 (local address 127.3.73.254:39487)
0807 10:51:00.230404 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:00.230411 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:00.230917 (+ 506us) server_negotiation.cc:407] Connection header received
0807 10:51:00.231110 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:00.231115 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:00.231188 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:00.231295 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:00.232260 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:00.233236 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:00.234859 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:00.235355 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:00.235473 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:00.247896 (+ 12423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.194:58785: BlockingRecv error: recv got EOF from 127.3.73.194:58785 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 07 10:51:00 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:00 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:00.607043 17047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:00.597985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.193:39933)
0807 10:51:00.598505 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:00.598523 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:00.598655 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:00.599030 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:00.599033 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:00.599428 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:00.599580 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:00.599589 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:00.600656 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:00.600660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:00.601730 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:00.601743 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:00.602591 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:00.602595 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:00.602725 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:00.603511 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:00.603538 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:00.606773 (+ 3235us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":274,"threads_started":1}
W20260807 10:51:00.607483 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:00.598102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:39933 (local address 127.3.73.254:39487)
0807 10:51:00.598381 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:00.598388 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:00.598637 (+ 249us) server_negotiation.cc:407] Connection header received
0807 10:51:00.598798 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:00.598802 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:00.598873 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:00.598974 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:00.599750 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:00.600523 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:00.601907 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:00.602435 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:00.602560 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:00.607102 (+ 4542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.193:39933: BlockingRecv error: recv got EOF from 127.3.73.193:39933 (error 108)
Metrics: {"server-negotiator.queue_time_us":123}
Aug 07 10:51:00 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:00 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:00.787762 17048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:00.778415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.195:33881)
0807 10:51:00.779017 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:00.779037 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:00.779168 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:00.779500 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:00.779504 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:00.779869 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:00.780070 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:00.780079 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:00.781166 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:00.781174 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:00.782359 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:00.782372 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:00.783280 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:00.783293 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:00.783504 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:00.784154 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:00.784172 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:00.787426 (+ 3254us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":373,"threads_started":1}
W20260807 10:51:00.788053 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:00.778469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:33881 (local address 127.3.73.254:39487)
0807 10:51:00.778741 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:00.778751 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:00.779134 (+ 383us) server_negotiation.cc:407] Connection header received
0807 10:51:00.779279 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:00.779284 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:00.779354 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:00.779441 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:00.780223 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:00.781005 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:00.782508 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:00.783123 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:00.783262 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:00.787731 (+ 4469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.195:33881: BlockingRecv error: recv got EOF from 127.3.73.195:33881 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Aug 07 10:51:01 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:01 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:01.258539 17050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:01.249086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.194:59417)
0807 10:51:01.249712 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:01.249729 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:01.249916 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:01.250319 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:01.250324 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:01.250774 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:01.250935 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:01.250944 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:01.252194 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:01.252200 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:01.253435 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:01.253447 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:01.254159 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:01.254165 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:01.254373 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:01.255038 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:01.255056 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:01.258170 (+ 3114us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":378,"threads_started":1}
W20260807 10:51:01.258643 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:01.249197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:59417 (local address 127.3.73.254:39487)
0807 10:51:01.249458 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:01.249467 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:01.249817 (+ 350us) server_negotiation.cc:407] Connection header received
0807 10:51:01.250098 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:01.250103 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:01.250170 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:01.250282 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:01.251083 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:01.252055 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:01.253584 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:01.254007 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:01.254145 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:01.258402 (+ 4257us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.194:59417: BlockingRecv error: recv got EOF from 127.3.73.194:59417 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 07 10:51:01 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:01 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:01.617727 17052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:01.608150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.193:35387)
0807 10:51:01.608980 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:01.608998 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:01.609173 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:01.609566 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:01.609569 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:01.610076 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:01.610259 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:01.610269 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:01.611366 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:01.611370 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:01.612430 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:01.612466 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:01.613384 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:01.613391 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:01.613540 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:01.614231 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:01.614251 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:01.617401 (+ 3150us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":451,"threads_started":1}
W20260807 10:51:01.618010 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:01.608286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:35387 (local address 127.3.73.254:39487)
0807 10:51:01.608469 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:01.608474 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:01.609109 (+ 635us) server_negotiation.cc:407] Connection header received
0807 10:51:01.609320 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:01.609326 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:01.609413 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:01.609521 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:01.610427 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:01.611223 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:01.612659 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:01.613198 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:01.613322 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:01.617698 (+ 4376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.193:35387: BlockingRecv error: recv got EOF from 127.3.73.193:35387 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 07 10:51:01 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:01 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:01.797865 17053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:01.788889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.195:52869)
0807 10:51:01.789492 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:01.789511 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:01.789639 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:01.790069 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:01.790073 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:01.790493 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:01.790657 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:01.790666 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:01.791778 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:01.791782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:01.792927 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:01.792942 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:01.793729 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:01.793734 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:01.793909 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:01.794625 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:01.794644 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:01.797616 (+ 2972us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":338,"threads_started":1}
W20260807 10:51:01.798190 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:01.789006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:52869 (local address 127.3.73.254:39487)
0807 10:51:01.789191 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:01.789197 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:01.789614 (+ 417us) server_negotiation.cc:407] Connection header received
0807 10:51:01.789830 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:01.789834 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:01.789904 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:01.789992 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:01.790800 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:01.791645 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:01.793093 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:01.793571 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:01.793679 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:01.797914 (+ 4235us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.195:52869: BlockingRecv error: recv got EOF from 127.3.73.195:52869 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 10:51:02 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:02 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:02.269272 17056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:02.259703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.194:48645)
0807 10:51:02.260416 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:02.260434 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:02.260609 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:02.261153 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:02.261158 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:02.261668 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:02.261867 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:02.261879 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:02.263115 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:02.263120 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:02.264124 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:02.264137 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:02.264893 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:02.264898 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:02.265037 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:02.265647 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:02.265666 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:02.268953 (+ 3287us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":353,"threads_started":1}
W20260807 10:51:02.269416 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:02.259993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:48645 (local address 127.3.73.254:39487)
0807 10:51:02.260252 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:02.260260 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:02.260533 (+ 273us) server_negotiation.cc:407] Connection header received
0807 10:51:02.260876 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:02.260883 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:02.260965 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:02.261082 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:02.262051 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:02.262988 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:02.264262 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:02.264669 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:02.264805 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:02.269190 (+ 4385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.194:48645: BlockingRecv error: recv got EOF from 127.3.73.194:48645 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 07 10:51:02 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:02 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:02.628703 17057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:02.618886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.193:36155)
0807 10:51:02.619487 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:02.619504 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:02.619623 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:02.620204 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:02.620209 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:02.620749 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:02.620973 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:02.620985 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:02.622186 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:02.622194 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:02.623105 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:02.623117 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:02.623872 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:02.623878 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:02.624061 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:02.624749 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:02.624837 (+ 88us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:02.628326 (+ 3489us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":325,"threads_started":1}
W20260807 10:51:02.628989 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:02.618953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:36155 (local address 127.3.73.254:39487)
0807 10:51:02.619164 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:02.619169 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:02.619599 (+ 430us) server_negotiation.cc:407] Connection header received
0807 10:51:02.619774 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:02.619778 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:02.619888 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:02.620000 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:02.621174 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:02.622026 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:02.623260 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:02.623696 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:02.623837 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:02.628672 (+ 4835us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.193:36155: BlockingRecv error: recv got EOF from 127.3.73.193:36155 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 07 10:51:02 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:02 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:02.808353 17058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:02.799217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.195:34335)
0807 10:51:02.800052 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:02.800076 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:02.800250 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:02.800624 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:02.800627 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:02.801097 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:02.801271 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:02.801280 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:02.802343 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:02.802347 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:02.803277 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:02.803290 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:02.804202 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:02.804206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:02.804433 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:02.805171 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:02.805190 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:02.808075 (+ 2885us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":444,"threads_started":1}
W20260807 10:51:02.808763 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:02.799314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:34335 (local address 127.3.73.254:39487)
0807 10:51:02.799623 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:02.799633 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:02.800174 (+ 541us) server_negotiation.cc:407] Connection header received
0807 10:51:02.800384 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:02.800389 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:02.800452 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:02.800534 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:02.801426 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:02.802183 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:02.803420 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:02.804056 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:02.804164 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:02.808313 (+ 4149us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.195:34335: BlockingRecv error: recv got EOF from 127.3.73.195:34335 (error 108)
Metrics: {"server-negotiator.queue_time_us":101}
Aug 07 10:51:03 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:03 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:03.280493 17060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:03.270525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.194:43463)
0807 10:51:03.271255 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:03.271273 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:03.271398 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:03.271765 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:03.271768 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:03.272312 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:03.272554 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:03.272566 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:03.274061 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:03.274066 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:03.275024 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:03.275037 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:03.275899 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:03.275908 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:03.276065 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:03.276835 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:03.276856 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:03.280119 (+ 3263us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":429,"threads_started":1}
W20260807 10:51:03.280704 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:03.270722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:43463 (local address 127.3.73.254:39487)
0807 10:51:03.270954 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:03.270961 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:03.271360 (+ 399us) server_negotiation.cc:407] Connection header received
0807 10:51:03.271515 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:03.271520 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:03.271595 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:03.271673 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:03.272715 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:03.273889 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:03.275167 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:03.275659 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:03.275766 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:03.280399 (+ 4633us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.194:43463: BlockingRecv error: recv got EOF from 127.3.73.194:43463 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 07 10:51:03 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:03 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:03.639964 17061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:03.629954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.193:35811)
0807 10:51:03.630712 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:03.630731 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:03.630913 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:03.631481 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:03.631485 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:03.632095 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:03.632367 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:03.632402 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:03.633720 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:03.633724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:03.634847 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:03.634861 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:03.635644 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:03.635648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:03.635776 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:03.636467 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:03.636487 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:03.639652 (+ 3165us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":345,"threads_started":1}
W20260807 10:51:03.640136 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:03.630106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:35811 (local address 127.3.73.254:39487)
0807 10:51:03.630391 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:03.630402 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:03.630828 (+ 426us) server_negotiation.cc:407] Connection header received
0807 10:51:03.631125 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:03.631135 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:03.631307 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:03.631427 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:03.632620 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:03.633562 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:03.634988 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:03.635492 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:03.635590 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:03.639892 (+ 4302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.193:35811: BlockingRecv error: recv got EOF from 127.3.73.193:35811 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 07 10:51:03 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:03 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:03.818548 17062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:03.809549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.195:60313)
0807 10:51:03.810235 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:03.810255 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:03.810449 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:03.810953 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:03.810957 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:03.811367 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:03.811536 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:03.811545 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:03.812686 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:03.812690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:03.813694 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:03.813706 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:03.814432 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:03.814437 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:03.814563 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:03.815255 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:03.815273 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:03.818311 (+ 3038us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":407,"threads_started":1}
W20260807 10:51:03.819032 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:03.809647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:60313 (local address 127.3.73.254:39487)
0807 10:51:03.809908 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:03.809918 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:03.810350 (+ 432us) server_negotiation.cc:407] Connection header received
0807 10:51:03.810612 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:03.810618 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:03.810734 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:03.810826 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:03.811702 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:03.812537 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:03.813856 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:03.814282 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:03.814382 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:03.818629 (+ 4247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.195:60313: BlockingRecv error: recv got EOF from 127.3.73.195:60313 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Aug 07 10:51:04 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:04 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:04.290745 17063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:04.281735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.194:48195)
0807 10:51:04.282465 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:04.282482 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:04.282649 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:04.283057 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:04.283062 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:04.283535 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:04.283732 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:04.283742 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:04.284788 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:04.284795 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:04.285741 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:04.285752 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:04.286533 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:04.286539 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:04.286714 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:04.287326 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:04.287344 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:04.290355 (+ 3011us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":624,"thread_start_us":382,"threads_started":1}
W20260807 10:51:04.290932 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:04.281851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:48195 (local address 127.3.73.254:39487)
0807 10:51:04.282080 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:04.282086 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:04.282571 (+ 485us) server_negotiation.cc:407] Connection header received
0807 10:51:04.282811 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:04.282816 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:04.282914 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:04.283025 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:04.283880 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:04.284614 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:04.285888 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:04.286356 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:04.286496 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:04.290646 (+ 4150us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.194:48195: BlockingRecv error: recv got EOF from 127.3.73.194:48195 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 07 10:51:04 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:04 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:04.651291 17067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:04.641260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.193:41649)
0807 10:51:04.641961 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:04.641985 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:04.642117 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:04.642557 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:04.642561 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:04.643019 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:04.643200 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:04.643209 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:04.644456 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:04.644461 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:04.645449 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:04.645462 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:04.646304 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:04.646310 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:04.646426 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:04.647195 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:04.647228 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:04.651047 (+ 3819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":346,"threads_started":1}
W20260807 10:51:04.651695 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:04.641349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:41649 (local address 127.3.73.254:39487)
0807 10:51:04.641542 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:04.641546 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:04.642090 (+ 544us) server_negotiation.cc:407] Connection header received
0807 10:51:04.642333 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:04.642337 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:04.642407 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:04.642499 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:04.643393 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:04.644276 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:04.645603 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:04.646110 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:04.646223 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:04.651317 (+ 5094us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.193:41649: BlockingRecv error: recv got EOF from 127.3.73.193:41649 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 07 10:51:04 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:04 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:04.829398 17068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:04.819553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.195:41435)
0807 10:51:04.820350 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:04.820366 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:04.820539 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:04.820947 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:04.820952 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:04.821535 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:04.821731 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:04.821741 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:04.822821 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:04.822833 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:04.823909 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:04.823922 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:04.824710 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:04.824739 (+ 29us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:04.824917 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:04.825571 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:04.825589 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:04.828924 (+ 3335us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":690,"thread_start_us":284,"threads_started":1}
W20260807 10:51:04.829515 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:04.819893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:41435 (local address 127.3.73.254:39487)
0807 10:51:04.820165 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:04.820171 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:04.820460 (+ 289us) server_negotiation.cc:407] Connection header received
0807 10:51:04.820668 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:04.820681 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:04.820772 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:04.820882 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:04.821891 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:04.822668 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:04.824137 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:04.824533 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:04.824676 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:04.829218 (+ 4542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.195:41435: BlockingRecv error: recv got EOF from 127.3.73.195:41435 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 07 10:51:05 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:05 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:05.307130 17069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:05.292036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.194:32971)
0807 10:51:05.292686 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:05.292706 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:05.292927 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:05.293353 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:05.293357 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:05.293810 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:05.293994 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:05.294005 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:05.295449 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:05.295454 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:05.296399 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:05.296412 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:05.297244 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:05.297249 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:05.297397 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:05.303529 (+ 6132us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:05.303553 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:05.306771 (+ 3218us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":308,"threads_started":1}
W20260807 10:51:05.307267 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:05.292139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:32971 (local address 127.3.73.254:39487)
0807 10:51:05.292436 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:05.292445 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:05.292872 (+ 427us) server_negotiation.cc:407] Connection header received
0807 10:51:05.293081 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:05.293087 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:05.293154 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:05.293230 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:05.294169 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:05.295284 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:05.296552 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:05.297050 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:05.297157 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:05.306998 (+ 9841us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.194:32971: BlockingRecv error: recv got EOF from 127.3.73.194:32971 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 07 10:51:05 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:05 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:05.662781 17070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:05.652478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.193:37757)
0807 10:51:05.653220 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:05.653239 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:05.653455 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:05.653859 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:05.653864 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:05.654367 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:05.654595 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:05.654609 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:05.656020 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:05.656026 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:05.657501 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:05.657517 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:05.658326 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:05.658332 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:05.658484 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:05.659168 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:05.659188 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:05.662499 (+ 3311us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":385,"threads_started":1}
W20260807 10:51:05.662976 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:05.652615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:37757 (local address 127.3.73.254:39487)
0807 10:51:05.652983 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:05.652992 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:05.653368 (+ 376us) server_negotiation.cc:407] Connection header received
0807 10:51:05.653603 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:05.653609 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:05.653693 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:05.653824 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:05.654786 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:05.655860 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:05.657664 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:05.658138 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:05.658276 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:05.662730 (+ 4454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.193:37757: BlockingRecv error: recv got EOF from 127.3.73.193:37757 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 07 10:51:05 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:05 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:05.840222 17072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:05.830571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.195:54639)
0807 10:51:05.831160 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:05.831226 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:05.831356 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:05.831665 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:05.831668 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:05.832074 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:05.832267 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:05.832276 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:05.833420 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:05.833424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:05.834375 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:05.834389 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:05.835207 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:05.835212 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:05.835331 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:05.836068 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:05.836095 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:05.839934 (+ 3839us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":333,"threads_started":1}
W20260807 10:51:05.840526 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:05.830664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:54639 (local address 127.3.73.254:39487)
0807 10:51:05.831010 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:05.831017 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:05.831313 (+ 296us) server_negotiation.cc:407] Connection header received
0807 10:51:05.831458 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:05.831462 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:05.831526 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:05.831608 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:05.832413 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:05.833266 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:05.834520 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:05.835030 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:05.835131 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:05.840192 (+ 5061us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.195:54639: BlockingRecv error: recv got EOF from 127.3.73.195:54639 (error 108)
Metrics: {"server-negotiator.queue_time_us":162}
Aug 07 10:51:06 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:06 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:06.318445 17074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:06.308474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.194:53027)
0807 10:51:06.309375 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:06.309393 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:06.309524 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:06.309978 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:06.309983 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:06.310466 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:06.310711 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:06.310727 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:06.311917 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:06.311922 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:06.313103 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:06.313119 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:06.313909 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:06.313914 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:06.314077 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:06.314743 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:06.314765 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:06.318165 (+ 3400us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":797,"thread_start_us":443,"threads_started":1}
W20260807 10:51:06.318751 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:06.308437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:53027 (local address 127.3.73.254:39487)
0807 10:51:06.308657 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:06.308664 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:06.309487 (+ 823us) server_negotiation.cc:407] Connection header received
0807 10:51:06.309633 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:06.309639 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:06.309744 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:06.309866 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:06.310912 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:06.311714 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:06.313281 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:06.313720 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:06.313828 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:06.318417 (+ 4589us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.194:53027: BlockingRecv error: recv got EOF from 127.3.73.194:53027 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 07 10:51:06 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:06 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:06.673362 17075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:06.663893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.193:45931)
0807 10:51:06.664547 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:06.664563 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:06.664756 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:06.665161 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:06.665165 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:06.665629 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:06.665781 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:06.665798 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:06.666876 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:06.666881 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:06.668211 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:06.668224 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:06.669293 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:06.669297 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:06.669406 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:06.670029 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:06.670047 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:06.673126 (+ 3079us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":378,"threads_started":1}
W20260807 10:51:06.673710 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:06.664081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:45931 (local address 127.3.73.254:39487)
0807 10:51:06.664362 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:06.664371 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:06.664662 (+ 291us) server_negotiation.cc:407] Connection header received
0807 10:51:06.664917 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:06.664923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:06.665009 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:06.665122 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:06.665950 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:06.666715 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:06.668378 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:06.668976 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:06.669099 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:06.673402 (+ 4303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.193:45931: BlockingRecv error: recv got EOF from 127.3.73.193:45931 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Aug 07 10:51:06 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:06 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:06.851665 17079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:06.841550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.195:42897)
0807 10:51:06.842234 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:06.842254 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:06.842390 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:06.842740 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:06.842745 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:06.843195 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:06.843408 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:06.843419 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:06.844525 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:06.844530 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:06.845808 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:06.845822 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:06.846736 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:06.846743 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:06.846927 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:06.847680 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:06.847701 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:06.851241 (+ 3540us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":397,"threads_started":1}
W20260807 10:51:06.851822 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:06.841833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:42897 (local address 127.3.73.254:39487)
0807 10:51:06.842107 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:06.842113 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:06.842340 (+ 227us) server_negotiation.cc:407] Connection header received
0807 10:51:06.842500 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:06.842504 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:06.842569 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:06.842655 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:06.843581 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:06.844367 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:06.845989 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:06.846526 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:06.846679 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:06.851535 (+ 4856us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.195:42897: BlockingRecv error: recv got EOF from 127.3.73.195:42897 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Aug 07 10:51:07 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:07 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:07.329437 17081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:07.319545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.194:39979)
0807 10:51:07.320174 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:07.320192 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:07.320317 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:07.320635 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:07.320639 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:07.321124 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:07.321351 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:07.321362 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:07.322613 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:07.322618 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:07.323658 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:07.323683 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:07.324407 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:07.324413 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:07.324548 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:07.325340 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:07.325360 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:07.329040 (+ 3680us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":362,"threads_started":1}
W20260807 10:51:07.329629 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:07.319625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:39979 (local address 127.3.73.254:39487)
0807 10:51:07.319795 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:07.319800 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:07.320276 (+ 476us) server_negotiation.cc:407] Connection header received
0807 10:51:07.320421 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:07.320425 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:07.320490 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:07.320575 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:07.321502 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:07.322436 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:07.323838 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:07.324231 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:07.324347 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:07.329321 (+ 4974us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.194:39979: BlockingRecv error: recv got EOF from 127.3.73.194:39979 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 07 10:51:07 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:07 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:07.683984 17082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:07.674453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.193:47821)
0807 10:51:07.675084 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:07.675127 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:07.675314 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:07.675716 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:07.675721 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:07.676345 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:07.676534 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:07.676543 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:07.677680 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:07.677686 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:07.678634 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:07.678647 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:07.679453 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:07.679459 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:07.679606 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:07.680298 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:07.680327 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:07.683671 (+ 3344us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":387,"threads_started":1}
W20260807 10:51:07.684194 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:07.674662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:47821 (local address 127.3.73.254:39487)
0807 10:51:07.674945 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:07.674951 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:07.675226 (+ 275us) server_negotiation.cc:407] Connection header received
0807 10:51:07.675452 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:07.675458 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:07.675540 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:07.675621 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:07.676695 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:07.677537 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:07.678797 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:07.679258 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:07.679381 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:07.683960 (+ 4579us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.193:47821: BlockingRecv error: recv got EOF from 127.3.73.193:47821 (error 108)
Metrics: {"server-negotiator.queue_time_us":149}
Aug 07 10:51:07 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:07 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:07.863055 17083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:07.853146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.195:60281)
0807 10:51:07.853750 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:07.853769 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:07.853992 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:07.854464 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:07.854469 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:07.854985 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:07.855146 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:07.855156 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:07.856262 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:07.856270 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:07.857276 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:07.857289 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:07.858080 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:07.858086 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:07.858290 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:07.859062 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:07.859082 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:07.862629 (+ 3547us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":515,"threads_started":1}
W20260807 10:51:07.863186 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:07.853376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:60281 (local address 127.3.73.254:39487)
0807 10:51:07.853590 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:07.853596 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:07.853910 (+ 314us) server_negotiation.cc:407] Connection header received
0807 10:51:07.854185 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:07.854190 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:07.854270 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:07.854413 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:07.855318 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:07.856102 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:07.857429 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:07.857908 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:07.858053 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:07.862889 (+ 4836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.195:60281: BlockingRecv error: recv got EOF from 127.3.73.195:60281 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 07 10:51:08 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:08 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:08.341529 17085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:08.330834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.194:48865)
0807 10:51:08.331470 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:08.331487 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:08.331668 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:08.332063 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:08.332068 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:08.332577 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:08.332884 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:08.332897 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:08.334233 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:08.334240 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:08.335514 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:08.335528 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:08.336513 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:08.336520 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:08.336774 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:08.337449 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:08.337471 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:08.341122 (+ 3651us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":349,"threads_started":1}
W20260807 10:51:08.341682 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:08.330957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:48865 (local address 127.3.73.254:39487)
0807 10:51:08.331248 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:08.331257 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:08.331593 (+ 336us) server_negotiation.cc:407] Connection header received
0807 10:51:08.331817 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:08.331829 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:08.331909 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:08.332027 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:08.333041 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:08.334078 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:08.335660 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:08.336195 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:08.336491 (+ 296us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:08.341374 (+ 4883us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.194:48865: BlockingRecv error: recv got EOF from 127.3.73.194:48865 (error 108)
Metrics: {"server-negotiator.queue_time_us":120}
Aug 07 10:51:08 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:08 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:08.694715 17087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:08.685207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.193:55115)
0807 10:51:08.686232 (+ 1025us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:08.686255 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:08.686404 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:08.686883 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:08.686887 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:08.687406 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:08.687568 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:08.687577 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:08.688661 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:08.688665 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:08.689715 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:08.689728 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:08.690425 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:08.690429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:08.690583 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:08.691286 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:08.691304 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:08.694359 (+ 3055us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":893,"thread_start_us":705,"threads_started":1}
W20260807 10:51:08.694918 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:08.685282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:55115 (local address 127.3.73.254:39487)
0807 10:51:08.685554 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:08.685564 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:08.686393 (+ 829us) server_negotiation.cc:407] Connection header received
0807 10:51:08.686550 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:08.686556 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:08.686652 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:08.686860 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:08.687731 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:08.688502 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:08.689878 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:08.690289 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:08.690387 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:08.694601 (+ 4214us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.193:55115: BlockingRecv error: recv got EOF from 127.3.73.193:55115 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 07 10:51:08 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:08 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:08.873687 17089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:08.864250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.195:34441)
0807 10:51:08.865008 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:08.865026 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:08.865217 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:08.865600 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:08.865605 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:08.866089 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:08.866275 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:08.866284 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:08.867355 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:08.867362 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:08.868319 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:08.868331 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:08.869171 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:08.869178 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:08.869359 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:08.870049 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:08.870068 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:08.873431 (+ 3363us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":334,"threads_started":1}
W20260807 10:51:08.873888 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:08.864390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:34441 (local address 127.3.73.254:39487)
0807 10:51:08.864702 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:08.864710 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:08.865148 (+ 438us) server_negotiation.cc:407] Connection header received
0807 10:51:08.865384 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:08.865388 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:08.865453 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:08.865564 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:08.866451 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:08.867197 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:08.868482 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:08.869001 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:08.869136 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:08.873604 (+ 4468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.195:34441: BlockingRecv error: recv got EOF from 127.3.73.195:34441 (error 108)
Metrics: {"server-negotiator.queue_time_us":126}
Aug 07 10:51:09 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:09 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:09.352412 17092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:09.342617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.194:49571)
0807 10:51:09.343218 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:09.343236 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:09.343357 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:09.343699 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:09.343702 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:09.344161 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:09.344347 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:09.344357 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:09.345541 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:09.345545 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:09.346541 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:09.346556 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:09.347289 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:09.347294 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:09.347453 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:09.348236 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:09.348256 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:09.352059 (+ 3803us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":356,"threads_started":1}
W20260807 10:51:09.352753 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:09.342716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:49571 (local address 127.3.73.254:39487)
0807 10:51:09.342973 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:09.342983 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:09.343321 (+ 338us) server_negotiation.cc:407] Connection header received
0807 10:51:09.343471 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:09.343475 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:09.343541 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:09.343626 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:09.344483 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:09.345362 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:09.346679 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:09.347143 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:09.347242 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:09.352400 (+ 5158us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.194:49571: BlockingRecv error: recv got EOF from 127.3.73.194:49571 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 07 10:51:09 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:09 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
W20260807 10:51:09.705488 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:09.695906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:44661 (local address 127.3.73.254:39487)
0807 10:51:09.696143 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:09.696152 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:09.696599 (+ 447us) server_negotiation.cc:407] Connection header received
0807 10:51:09.696858 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:09.696862 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:09.696925 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:09.697030 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:09.697870 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:09.698617 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:09.699887 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:09.700312 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:09.700458 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:09.705292 (+ 4834us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.193:44661: BlockingRecv error: recv got EOF from 127.3.73.193:44661 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 10:51:09.705968 17093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:09.695887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.193:44661)
0807 10:51:09.696485 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:09.696501 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:09.696672 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:09.697067 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:09.697072 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:09.697508 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:09.697671 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:09.697680 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:09.698765 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:09.698771 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:09.699731 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:09.699743 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:09.700509 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:09.700515 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:09.700681 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:09.701302 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:09.701332 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:09.705726 (+ 4394us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":300,"threads_started":1}
Aug 07 10:51:09 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:09 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:09.884473 17096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:09.874780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.195:50467)
0807 10:51:09.875375 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:09.875394 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:09.875521 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:09.875999 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:09.876003 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:09.876456 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:09.876621 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:09.876631 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:09.877811 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:09.877816 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:09.878818 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:09.878831 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:09.879818 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:09.879824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:09.879998 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:09.880933 (+ 935us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:09.880966 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:09.884212 (+ 3246us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":344,"threads_started":1}
W20260807 10:51:09.884769 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:09.875065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:50467 (local address 127.3.73.254:39487)
0807 10:51:09.875346 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:09.875354 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:09.875487 (+ 133us) server_negotiation.cc:407] Connection header received
0807 10:51:09.875740 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:09.875744 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:09.875818 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:09.875911 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:09.876809 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:09.877622 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:09.878991 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:09.879603 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:09.879775 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:09.884445 (+ 4670us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.195:50467: BlockingRecv error: recv got EOF from 127.3.73.195:50467 (error 108)
Metrics: {"server-negotiator.queue_time_us":118}
Aug 07 10:51:10 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:10 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:10.366633 17097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:10.353608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.194:48321)
0807 10:51:10.354297 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:10.354316 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:10.354498 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:10.354939 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:10.354943 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:10.355601 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:10.355780 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:10.355790 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:10.357340 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:10.357345 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:10.358289 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:10.358300 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:10.359066 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:10.359071 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:10.359214 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:10.362964 (+ 3750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:10.362988 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:10.366370 (+ 3382us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":365,"threads_started":1}
W20260807 10:51:10.366956 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:10.353833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:48321 (local address 127.3.73.254:39487)
0807 10:51:10.354020 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:10.354024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:10.354427 (+ 403us) server_negotiation.cc:407] Connection header received
0807 10:51:10.354642 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:10.354649 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:10.354722 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:10.354852 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:10.355979 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:10.357182 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:10.358437 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:10.358852 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:10.358975 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:10.366638 (+ 7663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.194:48321: BlockingRecv error: recv got EOF from 127.3.73.194:48321 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 07 10:51:10 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:10 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:10.717635 17099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:10.707043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.193:49925)
0807 10:51:10.707655 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:10.707672 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:10.707803 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:10.708197 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:10.708202 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:10.708652 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:10.708964 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:10.708976 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:10.710238 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:10.710244 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:10.711359 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:10.711376 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:10.712629 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:10.712634 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:10.712853 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:10.713610 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:10.713629 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:10.717302 (+ 3673us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":341,"threads_started":1}
W20260807 10:51:10.717857 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:10.707153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:49925 (local address 127.3.73.254:39487)
0807 10:51:10.707376 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:10.707381 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:10.707813 (+ 432us) server_negotiation.cc:407] Connection header received
0807 10:51:10.707927 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:10.707931 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:10.708003 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:10.708175 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:10.709164 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:10.710016 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:10.711616 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:10.712453 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:10.712596 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:10.717588 (+ 4992us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.193:49925: BlockingRecv error: recv got EOF from 127.3.73.193:49925 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 07 10:51:10 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:10 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:10.895776 17101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:10.885905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.195:49033)
0807 10:51:10.886562 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:10.886587 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:10.886755 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:10.887175 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:10.887179 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:10.887696 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:10.887885 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:10.887895 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:10.889279 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:10.889284 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:10.890294 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:10.890307 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:10.891155 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:10.891161 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:10.891292 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:10.892065 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:10.892096 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:10.895508 (+ 3412us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":290,"threads_started":1}
W20260807 10:51:10.896160 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:10.886144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:49033 (local address 127.3.73.254:39487)
0807 10:51:10.886524 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:10.886529 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:10.886688 (+ 159us) server_negotiation.cc:407] Connection header received
0807 10:51:10.886919 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:10.886923 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:10.886989 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:10.887118 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:10.888137 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:10.889104 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:10.890471 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:10.890959 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:10.891065 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:10.895860 (+ 4795us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.195:49033: BlockingRecv error: recv got EOF from 127.3.73.195:49033 (error 108)
Metrics: {"server-negotiator.queue_time_us":203}
Aug 07 10:51:11 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:11 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:11.377637 17102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:11.367823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.194:58469)
0807 10:51:11.368426 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:11.368443 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:11.368620 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:11.369018 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:11.369022 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:11.369460 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:11.369650 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:11.369659 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:11.370833 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:11.370846 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:11.371801 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:11.371813 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:11.372554 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:11.372560 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:11.372807 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:11.373792 (+ 985us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:11.373816 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:11.377258 (+ 3442us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":330,"threads_started":1}
W20260807 10:51:11.377760 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:11.367988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:58469 (local address 127.3.73.254:39487)
0807 10:51:11.368260 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:11.368268 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:11.368544 (+ 276us) server_negotiation.cc:407] Connection header received
0807 10:51:11.368787 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:11.368791 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:11.368865 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:11.368980 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:11.369816 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:11.370666 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:11.371968 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:11.372379 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:11.372540 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:11.377491 (+ 4951us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.194:58469: BlockingRecv error: recv got EOF from 127.3.73.194:58469 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Aug 07 10:51:11 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:11 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:11.729671 17103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:11.718851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.193:43699)
0807 10:51:11.719472 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:11.719499 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:11.719628 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:11.720066 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:11.720069 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:11.720609 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:11.720953 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:11.720968 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:11.722515 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:11.722520 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:11.723740 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:11.723756 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:11.724598 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:11.724603 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:11.724868 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:11.725606 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:11.725624 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:11.729248 (+ 3624us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":356,"threads_started":1}
W20260807 10:51:11.729758 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:11.719010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:43699 (local address 127.3.73.254:39487)
0807 10:51:11.719207 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:11.719211 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:11.719604 (+ 393us) server_negotiation.cc:407] Connection header received
0807 10:51:11.719800 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:11.719804 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:11.719875 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:11.719970 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:11.721209 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:11.722325 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:11.723932 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:11.724403 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:11.724515 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:11.729477 (+ 4962us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.193:43699: BlockingRecv error: recv got EOF from 127.3.73.193:43699 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 07 10:51:11 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:11 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.195@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:11.907282 17106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:11.897160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.195:36239)
0807 10:51:11.897712 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:11.897730 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:11.897875 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:11.898273 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:11.898277 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:11.898691 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:11.898923 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:11.898935 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:11.900219 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:11.900223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:11.901274 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:11.901287 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:11.902203 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:11.902210 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:11.902457 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:11.903380 (+ 923us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:11.903411 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:11.906997 (+ 3586us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":298,"threads_started":1}
W20260807 10:51:11.907671 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:11.897313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:36239 (local address 127.3.73.254:39487)
0807 10:51:11.897537 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:11.897543 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:11.897855 (+ 312us) server_negotiation.cc:407] Connection header received
0807 10:51:11.897990 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:11.897994 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:11.898066 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:11.898148 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:11.899124 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:11.900051 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:11.901448 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:11.901956 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:11.902095 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:11.907310 (+ 5215us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.195:36239: BlockingRecv error: recv got EOF from 127.3.73.195:36239 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 07 10:51:12 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
Aug 07 10:51:12 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.194@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:12.388845 17107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:12.379082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39487 (local address 127.3.73.194:54015)
0807 10:51:12.379689 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:12.379706 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:12.379835 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:12.380165 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:12.380168 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:12.380605 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:12.380906 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:12.380919 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:12.382006 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:12.382010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:12.382991 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:12.383004 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:12.383992 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:12.383999 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:12.384190 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:12.385055 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:12.385079 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:12.388472 (+ 3393us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.73.254:39487: Server oryx/127.3.73.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":340,"threads_started":1}
W20260807 10:51:12.389007 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:12.379221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:54015 (local address 127.3.73.254:39487)
0807 10:51:12.379450 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:12.379454 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:12.379803 (+ 349us) server_negotiation.cc:407] Connection header received
0807 10:51:12.379945 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:12.379949 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:12.380015 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:12.380107 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:12.381076 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:12.381854 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:12.383157 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:12.383754 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:12.383905 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:12.388700 (+ 4795us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.194:54015: BlockingRecv error: recv got EOF from 127.3.73.194:54015 (error 108)
Metrics: {"server-negotiator.queue_time_us":107}
I20260807 10:51:12.723026 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 16414
I20260807 10:51:12.739199 17110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:12.724243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51152 (local address 127.3.73.193:42481)
0807 10:51:12.724981 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:12.724988 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:12.725017 (+ 29us) server_negotiation.cc:407] Connection header received
0807 10:51:12.725080 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:12.725088 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:12.725321 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:12.725483 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:12.727143 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:12.728200 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:12.729556 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:12.730196 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:12.730372 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:12.733928 (+ 3556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:51:12.733938 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:51:12.733954 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:51:12.733990 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:51:12.738369 (+ 4379us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.3.73.193@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 10:51:12.738767 (+ 398us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51152: Request ticket server kudu/127.3.73.193@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":380,"threads_started":1}
W20260807 10:51:12.739238 3367 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 16414: Not authorized: Client connection negotiation failed: client connection to 127.3.73.193:42481: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.3.73.193@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 10:51:12.739341 17110 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51152: Request ticket server kudu/127.3.73.193@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 10:51:12.739506 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 16414
Aug 07 10:51:12 dist-test-slave-x5fp krb5kdc[15535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.73.193@KRBTEST.COM for oryx/127.3.73.254@KRBTEST.COM, Server not found in Kerberos database
I20260807 10:51:12.769665 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 16637
W20260807 10:51:12.769950 16830 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:12.730925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:35321 (local address 127.3.73.254:39487)
0807 10:51:12.731089 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:12.731093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:12.731450 (+ 357us) server_negotiation.cc:407] Connection header received
0807 10:51:12.731600 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:12.731606 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:12.731697 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:12.731858 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:12.732787 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:12.733947 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:12.736110 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:12.736633 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:12.736821 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:12.769618 (+ 32797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.73.193:35321: BlockingRecv error: recv got EOF from 127.3.73.193:35321 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 10:51:12.782783 17113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:12.770928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60902 (local address 127.3.73.194:35303)
0807 10:51:12.771552 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:12.771559 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:12.771597 (+ 38us) server_negotiation.cc:407] Connection header received
0807 10:51:12.771661 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:12.771668 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:12.771936 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:12.772143 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:12.773707 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:12.774778 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:12.776026 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:12.776607 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:12.776769 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:12.779374 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:51:12.779383 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:51:12.779398 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:51:12.779441 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:51:12.782074 (+ 2633us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.3.73.194@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 10:51:12.782423 (+ 349us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60902: Request ticket server kudu/127.3.73.194@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":277,"threads_started":1}
W20260807 10:51:12.782927 17113 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60902: Request ticket server kudu/127.3.73.194@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 10:51:12.782925 3367 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 16637: Not authorized: Client connection negotiation failed: client connection to 127.3.73.194:35303: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.3.73.194@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 10:51:12.783068 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 16637
I20260807 10:51:12.814280 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 16833
I20260807 10:51:12.828076 17114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:12.815521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50146 (local address 127.3.73.195:34517)
0807 10:51:12.816139 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:12.816147 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:12.816176 (+ 29us) server_negotiation.cc:407] Connection header received
0807 10:51:12.816269 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:12.816276 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:12.816516 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:12.816687 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:12.818279 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:12.819554 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:12.821001 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:12.821612 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:12.821728 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:12.824554 (+ 2826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:51:12.824563 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:51:12.824579 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:51:12.824632 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:51:12.827531 (+ 2899us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.3.73.195@KRBTEST.COM not found in keytab (ticket kvno 2)
0807 10:51:12.827811 (+ 280us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50146: Request ticket server kudu/127.3.73.195@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":314,"threads_started":1}
W20260807 10:51:12.828226 17114 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50146: Request ticket server kudu/127.3.73.195@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260807 10:51:12.828348 3367 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 16833: Not authorized: Client connection negotiation failed: client connection to 127.3.73.195:34517: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.3.73.195@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260807 10:51:12.828568 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 16833
I20260807 10:51:12.851744 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 16298
I20260807 10:51:12.853288 16402 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:51:12.978461 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 16298
2026-08-07T10:51:13Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (22321 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T10:51:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:51:13Z Disabled control of system clock
I20260807 10:51:13.086972 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:36037
--webserver_interface=127.3.73.254
--webserver_port=0
--builtin_ntp_servers=127.3.73.212:37877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:36037
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 10:51:13.444981 17121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:51:13.445302 17121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:51:13.445446 17121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:51:13.452806 17121 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:51:13.452939 17121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:51:13.453029 17121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:51:13.453125 17121 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:51:13.453217 17121 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:51:13.462760 17121 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:37877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:36037
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:36037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:51:13.464200 17121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:51:13.466097 17121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:51:13.479144 17132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:51:13.479142 17134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:51:13.479169 17136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:51:13.479631 17121 server_base.cc:1034] running on GCE node
I20260807 10:51:13.480944 17121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:51:13.482489 17121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:51:13.483741 17121 hybrid_clock.cc:648] HybridClock initialized: now 1786099873483677 us; error 58 us; skew 500 ppm
I20260807 10:51:13.484117 17121 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 10:51:13.486606 17121 webserver.cc:460] Webserver started at http://127.3.73.254:34911/ using document root <none> and password file <none>
I20260807 10:51:13.487329 17121 fs_manager.cc:359] Metadata directory not provided
I20260807 10:51:13.487464 17121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:51:13.487825 17121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:51:13.490816 17121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "4d38c6c77ab9414984f3dba2833e79b7"
format_stamp: "Formatted at 2026-08-07 10:51:13 on dist-test-slave-x5fp"
server_key: "9608cb39b1d3fbee1ebdc2213593eadc"
server_key_iv: "a0c50c49ee085f2fe3d9ef2cbdd4f766"
server_key_version: "encryptionkey@0"
I20260807 10:51:13.491621 17121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "4d38c6c77ab9414984f3dba2833e79b7"
format_stamp: "Formatted at 2026-08-07 10:51:13 on dist-test-slave-x5fp"
server_key: "9608cb39b1d3fbee1ebdc2213593eadc"
server_key_iv: "a0c50c49ee085f2fe3d9ef2cbdd4f766"
server_key_version: "encryptionkey@0"
I20260807 10:51:13.497092 17121 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260807 10:51:13.501111 17145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:51:13.502625 17121 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 10:51:13.502956 17121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "4d38c6c77ab9414984f3dba2833e79b7"
format_stamp: "Formatted at 2026-08-07 10:51:13 on dist-test-slave-x5fp"
server_key: "9608cb39b1d3fbee1ebdc2213593eadc"
server_key_iv: "a0c50c49ee085f2fe3d9ef2cbdd4f766"
server_key_version: "encryptionkey@0"
I20260807 10:51:13.503218 17121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:51:13.514792 17121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:51:13.515741 17121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:51:13.516083 17121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:51:13.537086 17121 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:36037
I20260807 10:51:13.537146 17226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:36037 every 8 connection(s)
I20260807 10:51:13.538594 17121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:51:13.539635 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 17121
I20260807 10:51:13.539849 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:51:13.540593 3367 external_mini_cluster.cc:1428] Setting key bc22e1139bf9d1c43497e80b1fb9c0f6
I20260807 10:51:13.543368 17227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:51:13.555277 17231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:13.542799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59036 (local address 127.3.73.254:36037)
0807 10:51:13.543922 (+ 1123us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:13.543933 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:13.543977 (+ 44us) server_negotiation.cc:407] Connection header received
0807 10:51:13.545728 (+ 1751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:13.545751 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:13.546223 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:13.546659 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:13.547427 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:13.548951 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:13.550338 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:13.551050 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:13.551262 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:13.552674 (+ 1412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:51:13.552681 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:51:13.552697 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:51:13.553400 (+ 703us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:51:13.553485 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:51:13.553514 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:51:13.553649 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 10:51:13.553905 (+ 256us) server_negotiation.cc:299] Negotiation successful
0807 10:51:13.554235 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":393,"threads_started":1}
I20260807 10:51:13.555383 17227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d38c6c77ab9414984f3dba2833e79b7: Bootstrap starting.
I20260807 10:51:13.559401 17227 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d38c6c77ab9414984f3dba2833e79b7: Neither blocks nor log segments found. Creating new log.
I20260807 10:51:13.560505 17227 log.cc:824] T 00000000000000000000000000000000 P 4d38c6c77ab9414984f3dba2833e79b7: Log is configured to *not* fsync() on all Append() calls
I20260807 10:51:13.564038 17227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d38c6c77ab9414984f3dba2833e79b7: No bootstrap required, opened a new log
I20260807 10:51:13.572429 17227 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d38c6c77ab9414984f3dba2833e79b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d38c6c77ab9414984f3dba2833e79b7" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 36037 } }
I20260807 10:51:13.572997 17227 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d38c6c77ab9414984f3dba2833e79b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:51:13.573084 17227 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d38c6c77ab9414984f3dba2833e79b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d38c6c77ab9414984f3dba2833e79b7, State: Initialized, Role: FOLLOWER
I20260807 10:51:13.573632 17227 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d38c6c77ab9414984f3dba2833e79b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d38c6c77ab9414984f3dba2833e79b7" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 36037 } }
I20260807 10:51:13.573863 17227 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d38c6c77ab9414984f3dba2833e79b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:51:13.573973 17227 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d38c6c77ab9414984f3dba2833e79b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:51:13.574103 17227 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d38c6c77ab9414984f3dba2833e79b7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:51:13.576388 17227 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d38c6c77ab9414984f3dba2833e79b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d38c6c77ab9414984f3dba2833e79b7" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 36037 } }
I20260807 10:51:13.577001 17227 leader_election.cc:302] T 00000000000000000000000000000000 P 4d38c6c77ab9414984f3dba2833e79b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d38c6c77ab9414984f3dba2833e79b7; no voters:
I20260807 10:51:13.577642 17227 leader_election.cc:288] T 00000000000000000000000000000000 P 4d38c6c77ab9414984f3dba2833e79b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:51:13.577795 17233 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d38c6c77ab9414984f3dba2833e79b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:51:13.578668 17233 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d38c6c77ab9414984f3dba2833e79b7 [term 1 LEADER]: Becoming Leader. State: Replica: 4d38c6c77ab9414984f3dba2833e79b7, State: Running, Role: LEADER
I20260807 10:51:13.579599 17233 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d38c6c77ab9414984f3dba2833e79b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d38c6c77ab9414984f3dba2833e79b7" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 36037 } }
I20260807 10:51:13.580307 17227 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d38c6c77ab9414984f3dba2833e79b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:51:13.583061 17235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d38c6c77ab9414984f3dba2833e79b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d38c6c77ab9414984f3dba2833e79b7. Latest consensus state: current_term: 1 leader_uuid: "4d38c6c77ab9414984f3dba2833e79b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d38c6c77ab9414984f3dba2833e79b7" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 36037 } } }
I20260807 10:51:13.583278 17234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d38c6c77ab9414984f3dba2833e79b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d38c6c77ab9414984f3dba2833e79b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d38c6c77ab9414984f3dba2833e79b7" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 36037 } } }
I20260807 10:51:13.583654 17235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d38c6c77ab9414984f3dba2833e79b7 [sys.catalog]: This master's current role is: LEADER
I20260807 10:51:13.583725 17234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d38c6c77ab9414984f3dba2833e79b7 [sys.catalog]: This master's current role is: LEADER
I20260807 10:51:13.588449 17245 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:51:13.592976 17245 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:51:13.602744 17245 catalog_manager.cc:1347] Generated new cluster ID: 7d0461249e1f423cb36179e619f84dc5
I20260807 10:51:13.602895 17245 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:51:13.617884 17245 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:51:13.619855 17245 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:51:13.627923 17245 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d38c6c77ab9414984f3dba2833e79b7: Generated new TSK 0
I20260807 10:51:13.629297 17245 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 10:51:13.634893 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.193:0
--local_ip_for_outbound_sockets=127.3.73.193
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:36037
--builtin_ntp_servers=127.3.73.212:37877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 10:51:14.009707 17263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:51:14.010061 17263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:51:14.010228 17263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:51:14.021821 17263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:51:14.021937 17263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:51:14.022045 17263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.193
I20260807 10:51:14.031371 17263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:37877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:36037
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.193
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:51:14.032697 17263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:51:14.034544 17263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:51:14.048609 17274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:51:14.048780 17271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:51:14.048810 17269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:51:14.050014 17263 server_base.cc:1034] running on GCE node
I20260807 10:51:14.050653 17263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:51:14.051697 17263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:51:14.052995 17263 hybrid_clock.cc:648] HybridClock initialized: now 1786099874052922 us; error 58 us; skew 500 ppm
I20260807 10:51:14.053391 17263 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 10:51:14.056267 17263 webserver.cc:460] Webserver started at http://127.3.73.193:34747/ using document root <none> and password file <none>
I20260807 10:51:14.057152 17263 fs_manager.cc:359] Metadata directory not provided
I20260807 10:51:14.057300 17263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:51:14.057665 17263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:51:14.060881 17263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/data/instance:
uuid: "23ec4c9057b7424a84cbfd39f00d106f"
format_stamp: "Formatted at 2026-08-07 10:51:14 on dist-test-slave-x5fp"
server_key: "3a5a852bfd600b0ea05b91eec907cdc9"
server_key_iv: "694da658a912c1dd442103c3c3ed3d13"
server_key_version: "encryptionkey@0"
I20260807 10:51:14.061872 17263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance:
uuid: "23ec4c9057b7424a84cbfd39f00d106f"
format_stamp: "Formatted at 2026-08-07 10:51:14 on dist-test-slave-x5fp"
server_key: "3a5a852bfd600b0ea05b91eec907cdc9"
server_key_iv: "694da658a912c1dd442103c3c3ed3d13"
server_key_version: "encryptionkey@0"
I20260807 10:51:14.068419 17263 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 10:51:14.072595 17284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:51:14.074587 17263 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 10:51:14.074857 17263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/wal
uuid: "23ec4c9057b7424a84cbfd39f00d106f"
format_stamp: "Formatted at 2026-08-07 10:51:14 on dist-test-slave-x5fp"
server_key: "3a5a852bfd600b0ea05b91eec907cdc9"
server_key_iv: "694da658a912c1dd442103c3c3ed3d13"
server_key_version: "encryptionkey@0"
I20260807 10:51:14.075148 17263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:51:14.109127 17263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:51:14.110139 17263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:51:14.110630 17263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:51:14.111919 17263 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:51:14.113819 17263 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:51:14.113970 17263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:51:14.114149 17263 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:51:14.114270 17263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:51:14.156381 17263 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.193:43173
I20260807 10:51:14.156440 17453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.193:43173 every 8 connection(s)
I20260807 10:51:14.158022 17263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
I20260807 10:51:14.159621 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 17263
I20260807 10:51:14.159837 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance
I20260807 10:51:14.160764 3367 external_mini_cluster.cc:1428] Setting key 1070af01d74a21248a71bbc4e32de7e3
I20260807 10:51:14.163978 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.194:0
--local_ip_for_outbound_sockets=127.3.73.194
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:36037
--builtin_ntp_servers=127.3.73.212:37877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 10:51:14.173431 17456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:14.161499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:41153 (local address 127.3.73.254:36037)
0807 10:51:14.161981 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:14.161988 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:14.163342 (+ 1354us) server_negotiation.cc:407] Connection header received
0807 10:51:14.165335 (+ 1993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:14.165344 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:14.165452 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:14.165581 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:14.166910 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:14.168032 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:14.169866 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:14.170588 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:14.170735 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:14.172290 (+ 1555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:51:14.172299 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:51:14.172307 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:51:14.172336 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:51:14.172405 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:51:14.172436 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:51:14.172533 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 10:51:14.172822 (+ 289us) server_negotiation.cc:299] Negotiation successful
0807 10:51:14.172944 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":251,"threads_started":1}
I20260807 10:51:14.174445 17457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:14.162030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:36037 (local address 127.3.73.193:41153)
0807 10:51:14.163173 (+ 1143us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:14.163221 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:14.165051 (+ 1830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:14.165913 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:14.165922 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:14.166023 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:14.166693 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:14.166713 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:14.168227 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:14.168231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:14.169705 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:14.169719 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:14.170806 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:14.170811 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:14.171061 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:14.171913 (+ 852us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 10:51:14.171916 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:14.171959 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 10:51:14.172044 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:51:14.172559 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:51:14.172569 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:51:14.172576 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 10:51:14.172807 (+ 231us) client_negotiation.cc:240] Negotiation successful
0807 10:51:14.173138 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":385,"threads_started":1}
I20260807 10:51:14.176188 17454 heartbeater.cc:347] Connected to a master server at 127.3.73.254:36037
I20260807 10:51:14.176551 17454 heartbeater.cc:464] Registering TS with master...
I20260807 10:51:14.177527 17454 heartbeater.cc:511] Master 127.3.73.254:36037 requested a full tablet report, sending...
I20260807 10:51:14.179800 17173 ts_manager.cc:194] Registered new tserver with Master: 23ec4c9057b7424a84cbfd39f00d106f (127.3.73.193:43173)
I20260807 10:51:14.181540 17173 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.73.193:41153
W20260807 10:51:14.518711 17458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:51:14.519049 17458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:51:14.519227 17458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:51:14.526482 17458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:51:14.526626 17458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:51:14.526808 17458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.194
I20260807 10:51:14.535853 17458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:37877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:36037
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.194
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:51:14.537314 17458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:51:14.539083 17458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:51:14.552795 17465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:51:14.552830 17464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:51:14.553516 17467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:51:14.554507 17458 server_base.cc:1034] running on GCE node
I20260807 10:51:14.555302 17458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:51:14.556412 17458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:51:14.557655 17458 hybrid_clock.cc:648] HybridClock initialized: now 1786099874557572 us; error 75 us; skew 500 ppm
I20260807 10:51:14.558038 17458 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 10:51:14.560483 17458 webserver.cc:460] Webserver started at http://127.3.73.194:34547/ using document root <none> and password file <none>
I20260807 10:51:14.561309 17458 fs_manager.cc:359] Metadata directory not provided
I20260807 10:51:14.561430 17458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:51:14.561780 17458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:51:14.564816 17458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/data/instance:
uuid: "639c59b95b8d434292afc353acacdf1c"
format_stamp: "Formatted at 2026-08-07 10:51:14 on dist-test-slave-x5fp"
server_key: "2f882870eefa52b29e16b42a1cc8238c"
server_key_iv: "35cb4fda4cdfc4fc5b53486714b1bc2d"
server_key_version: "encryptionkey@0"
I20260807 10:51:14.565732 17458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance:
uuid: "639c59b95b8d434292afc353acacdf1c"
format_stamp: "Formatted at 2026-08-07 10:51:14 on dist-test-slave-x5fp"
server_key: "2f882870eefa52b29e16b42a1cc8238c"
server_key_iv: "35cb4fda4cdfc4fc5b53486714b1bc2d"
server_key_version: "encryptionkey@0"
I20260807 10:51:14.571184 17458 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 10:51:14.574756 17474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:51:14.576264 17458 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 10:51:14.576444 17458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/wal
uuid: "639c59b95b8d434292afc353acacdf1c"
format_stamp: "Formatted at 2026-08-07 10:51:14 on dist-test-slave-x5fp"
server_key: "2f882870eefa52b29e16b42a1cc8238c"
server_key_iv: "35cb4fda4cdfc4fc5b53486714b1bc2d"
server_key_version: "encryptionkey@0"
I20260807 10:51:14.576691 17458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:51:14.589875 17458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:51:14.590713 17458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:51:14.591037 17458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:51:14.592039 17458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:51:14.593688 17458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:51:14.593829 17458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:51:14.593961 17458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:51:14.594048 17458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:51:14.631245 17458 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.194:36247
I20260807 10:51:14.631357 17641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.194:36247 every 8 connection(s)
I20260807 10:51:14.632591 17458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
I20260807 10:51:14.638777 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 17458
I20260807 10:51:14.638952 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance
I20260807 10:51:14.639436 3367 external_mini_cluster.cc:1428] Setting key 05a2025ac4d07898b43c9e0036e209a6
I20260807 10:51:14.642418 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.195:0
--local_ip_for_outbound_sockets=127.3.73.195
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:36037
--builtin_ntp_servers=127.3.73.212:37877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 10:51:14.646236 17456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:14.635319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:32835 (local address 127.3.73.254:36037)
0807 10:51:14.635503 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:14.635507 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:14.636837 (+ 1330us) server_negotiation.cc:407] Connection header received
0807 10:51:14.638703 (+ 1866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:14.638710 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:14.638808 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:14.638948 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:14.640109 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:14.641029 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:14.642900 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:14.643610 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:14.643764 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:14.645241 (+ 1477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:51:14.645249 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:51:14.645257 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:51:14.645284 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:51:14.645333 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:51:14.645352 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:51:14.645454 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 10:51:14.645719 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 10:51:14.645833 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 10:51:14.647181 17645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:14.635724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:36037 (local address 127.3.73.194:32835)
0807 10:51:14.636618 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:14.636664 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:14.638429 (+ 1765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:14.639195 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:14.639203 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:14.639292 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:14.639932 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:14.639949 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:14.641192 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:14.641205 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:14.642731 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:14.642745 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:14.643845 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:14.643850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:14.644064 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:14.644898 (+ 834us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 10:51:14.644900 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:14.644943 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 10:51:14.645035 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:51:14.645494 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:51:14.645504 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:51:14.645511 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 10:51:14.645679 (+ 168us) client_negotiation.cc:240] Negotiation successful
0807 10:51:14.645992 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":276,"threads_started":1}
I20260807 10:51:14.648761 17642 heartbeater.cc:347] Connected to a master server at 127.3.73.254:36037
I20260807 10:51:14.649077 17642 heartbeater.cc:464] Registering TS with master...
I20260807 10:51:14.649860 17642 heartbeater.cc:511] Master 127.3.73.254:36037 requested a full tablet report, sending...
I20260807 10:51:14.651382 17173 ts_manager.cc:194] Registered new tserver with Master: 639c59b95b8d434292afc353acacdf1c (127.3.73.194:36247)
I20260807 10:51:14.652355 17173 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.73.194:32835
W20260807 10:51:14.996016 17646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:51:14.996302 17646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:51:14.996452 17646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:51:15.003932 17646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:51:15.004058 17646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:51:15.004240 17646 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.195
I20260807 10:51:15.014142 17646 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:37877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:36037
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.195
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:51:15.015578 17646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:51:15.017567 17646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:51:15.031666 17655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:51:15.031724 17659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:51:15.031790 17656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:51:15.033483 17646 server_base.cc:1034] running on GCE node
I20260807 10:51:15.034420 17646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:51:15.035544 17646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:51:15.036850 17646 hybrid_clock.cc:648] HybridClock initialized: now 1786099875036790 us; error 52 us; skew 500 ppm
I20260807 10:51:15.037226 17646 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 10:51:15.039968 17646 webserver.cc:460] Webserver started at http://127.3.73.195:35455/ using document root <none> and password file <none>
I20260807 10:51:15.040807 17646 fs_manager.cc:359] Metadata directory not provided
I20260807 10:51:15.040935 17646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:51:15.041319 17646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:51:15.044366 17646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/data/instance:
uuid: "1899c4a0ba2c42c48be916a73d7bdaa0"
format_stamp: "Formatted at 2026-08-07 10:51:15 on dist-test-slave-x5fp"
server_key: "c8cce49760ab5bcc53c325e42b1112b9"
server_key_iv: "49b30ce76ba24edd2a5ee7aedd5a3197"
server_key_version: "encryptionkey@0"
I20260807 10:51:15.045293 17646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance:
uuid: "1899c4a0ba2c42c48be916a73d7bdaa0"
format_stamp: "Formatted at 2026-08-07 10:51:15 on dist-test-slave-x5fp"
server_key: "c8cce49760ab5bcc53c325e42b1112b9"
server_key_iv: "49b30ce76ba24edd2a5ee7aedd5a3197"
server_key_version: "encryptionkey@0"
I20260807 10:51:15.050988 17646 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 10:51:15.054793 17667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:51:15.056604 17646 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 10:51:15.056887 17646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/wal
uuid: "1899c4a0ba2c42c48be916a73d7bdaa0"
format_stamp: "Formatted at 2026-08-07 10:51:15 on dist-test-slave-x5fp"
server_key: "c8cce49760ab5bcc53c325e42b1112b9"
server_key_iv: "49b30ce76ba24edd2a5ee7aedd5a3197"
server_key_version: "encryptionkey@0"
I20260807 10:51:15.057159 17646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:51:15.084980 17646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:51:15.086079 17646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:51:15.086426 17646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:51:15.087483 17646 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:51:15.089099 17646 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:51:15.089193 17646 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:51:15.089274 17646 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:51:15.089335 17646 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:51:15.127425 17646 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.195:46649
I20260807 10:51:15.127532 17840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.195:46649 every 8 connection(s)
I20260807 10:51:15.129000 17646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
I20260807 10:51:15.137250 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 17646
I20260807 10:51:15.137439 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance
I20260807 10:51:15.137975 3367 external_mini_cluster.cc:1428] Setting key e2e6cebd4a8171e679e90fce013b3893
I20260807 10:51:15.144105 17456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:15.131969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:36143 (local address 127.3.73.254:36037)
0807 10:51:15.132221 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:15.132226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:15.133757 (+ 1531us) server_negotiation.cc:407] Connection header received
0807 10:51:15.135650 (+ 1893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:15.135655 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:15.135730 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:15.135837 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:15.137149 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:15.138198 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:15.140658 (+ 2460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:15.141454 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:15.141602 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:15.143251 (+ 1649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:51:15.143257 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:51:15.143261 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:51:15.143282 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:51:15.143322 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:51:15.143335 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:51:15.143424 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 10:51:15.143743 (+ 319us) server_negotiation.cc:299] Negotiation successful
0807 10:51:15.143836 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 10:51:15.145397 17843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:15.132510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:36037 (local address 127.3.73.195:36143)
0807 10:51:15.133572 (+ 1062us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:15.133621 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:15.135319 (+ 1698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:15.136088 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:15.136096 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:15.136194 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:15.136923 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:15.136954 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:15.138389 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:15.138395 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:15.140402 (+ 2007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:15.140416 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:15.141750 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:15.141760 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:15.142051 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:15.142821 (+ 770us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 10:51:15.142824 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:15.142881 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 10:51:15.142985 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:51:15.143452 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:51:15.143462 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:51:15.143470 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 10:51:15.143655 (+ 185us) client_negotiation.cc:240] Negotiation successful
0807 10:51:15.144032 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":385,"threads_started":1}
I20260807 10:51:15.147244 17841 heartbeater.cc:347] Connected to a master server at 127.3.73.254:36037
I20260807 10:51:15.147668 17841 heartbeater.cc:464] Registering TS with master...
I20260807 10:51:15.148659 17841 heartbeater.cc:511] Master 127.3.73.254:36037 requested a full tablet report, sending...
I20260807 10:51:15.150457 17173 ts_manager.cc:194] Registered new tserver with Master: 1899c4a0ba2c42c48be916a73d7bdaa0 (127.3.73.195:46649)
I20260807 10:51:15.151366 17173 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.73.195:36143
I20260807 10:51:15.154549 3367 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 10:51:15.162501 17456 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:15.160180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59042 (local address 127.3.73.254:36037)
0807 10:51:15.160482 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:15.160487 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:15.161171 (+ 684us) server_negotiation.cc:407] Connection header received
0807 10:51:15.161404 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:15.161410 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:15.161521 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:15.161639 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:15.162217 (+ 578us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59042: BlockingRecv error: recv got EOF from 127.0.0.1:59042 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 10:51:15.163981 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 17263
I20260807 10:51:15.173727 17857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:15.165101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50744 (local address 127.3.73.193:43173)
0807 10:51:15.165715 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:15.165721 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:15.165751 (+ 30us) server_negotiation.cc:407] Connection header received
0807 10:51:15.165851 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:15.165858 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:15.166077 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:15.166245 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:15.166704 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:15.167747 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:15.169426 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:15.170495 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:15.170660 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:15.172016 (+ 1356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:51:15.172026 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:51:15.172041 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:51:15.172671 (+ 630us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:51:15.172783 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:51:15.172810 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:51:15.172987 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 10:51:15.173153 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 10:51:15.173313 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":333,"threads_started":1}
I20260807 10:51:15.175145 17449 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:51:15.258351 17454 heartbeater.cc:503] Master 127.3.73.254:36037 was elected leader, sending a full tablet report...
I20260807 10:51:15.259654 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 17263
W20260807 10:51:15.287147 17150 connection.cc:535] server connection from 127.3.73.193:41153 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 10:51:15.287400 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 17458
I20260807 10:51:15.295779 17859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:15.288364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57662 (local address 127.3.73.194:36247)
0807 10:51:15.288996 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:15.289004 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:15.289032 (+ 28us) server_negotiation.cc:407] Connection header received
0807 10:51:15.289094 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:15.289100 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:15.289335 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:15.289512 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:15.289956 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:15.291024 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:15.292254 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:15.292934 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:15.293078 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:15.294240 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:51:15.294249 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:51:15.294264 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:51:15.294899 (+ 635us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:51:15.294951 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:51:15.294972 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:51:15.295105 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 10:51:15.295260 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 10:51:15.295430 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":271,"threads_started":1}
I20260807 10:51:15.297164 17635 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:51:15.381798 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 17458
I20260807 10:51:15.411407 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 17646
I20260807 10:51:15.425278 17861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:15.412615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56594 (local address 127.3.73.195:46649)
0807 10:51:15.413399 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:15.413406 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:15.413438 (+ 32us) server_negotiation.cc:407] Connection header received
0807 10:51:15.413502 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:15.413509 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:15.413755 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:15.413892 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:15.414368 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:15.415825 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:15.417838 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:15.418564 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:15.418755 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:15.423418 (+ 4663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:51:15.423429 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:51:15.423445 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:51:15.424146 (+ 701us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:51:15.424204 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:51:15.424237 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:51:15.424406 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 10:51:15.424542 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 10:51:15.424698 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":425,"threads_started":1}
I20260807 10:51:15.426837 17835 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:51:15.513325 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 17646
I20260807 10:51:15.538973 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 17121
I20260807 10:51:15.540340 17220 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:51:15.668164 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 17121
2026-08-07T10:51:15Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2684 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T10:51:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:51:15Z Disabled control of system clock
I20260807 10:51:15.775570 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:34105
--webserver_interface=127.3.73.254
--webserver_port=0
--builtin_ntp_servers=127.3.73.212:35125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:34105
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 10:51:16.141530 17868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:51:16.141845 17868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:51:16.142014 17868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:51:16.149618 17868 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:51:16.149744 17868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:51:16.149835 17868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:51:16.149953 17868 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:51:16.150049 17868 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:51:16.159673 17868 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:34105
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:34105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:51:16.161172 17868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:51:16.163058 17868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:51:16.175714 17879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:51:16.175625 17880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:51:16.175690 17882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:51:16.176817 17868 server_base.cc:1034] running on GCE node
I20260807 10:51:16.177775 17868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:51:16.179379 17868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:51:16.180599 17868 hybrid_clock.cc:648] HybridClock initialized: now 1786099876180526 us; error 68 us; skew 500 ppm
I20260807 10:51:16.181038 17868 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 10:51:16.183498 17868 webserver.cc:460] Webserver started at http://127.3.73.254:45875/ using document root <none> and password file <none>
I20260807 10:51:16.184257 17868 fs_manager.cc:359] Metadata directory not provided
I20260807 10:51:16.184377 17868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:51:16.184816 17868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:51:16.187965 17868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "6bc08c05f5b345acbec30b749cf41afd"
format_stamp: "Formatted at 2026-08-07 10:51:16 on dist-test-slave-x5fp"
server_key: "28d96fe5bef11effcccfd0b1bfd77726"
server_key_iv: "d7505216f027f122069e3ac3ee9d4134"
server_key_version: "encryptionkey@0"
I20260807 10:51:16.188856 17868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "6bc08c05f5b345acbec30b749cf41afd"
format_stamp: "Formatted at 2026-08-07 10:51:16 on dist-test-slave-x5fp"
server_key: "28d96fe5bef11effcccfd0b1bfd77726"
server_key_iv: "d7505216f027f122069e3ac3ee9d4134"
server_key_version: "encryptionkey@0"
I20260807 10:51:16.194352 17868 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 10:51:16.198040 17891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:51:16.199435 17868 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 10:51:16.199633 17868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "6bc08c05f5b345acbec30b749cf41afd"
format_stamp: "Formatted at 2026-08-07 10:51:16 on dist-test-slave-x5fp"
server_key: "28d96fe5bef11effcccfd0b1bfd77726"
server_key_iv: "d7505216f027f122069e3ac3ee9d4134"
server_key_version: "encryptionkey@0"
I20260807 10:51:16.199908 17868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:51:16.205950 17868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:51:16.206779 17868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:51:16.207103 17868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:51:16.227504 17868 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:34105
I20260807 10:51:16.227530 17971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:34105 every 8 connection(s)
I20260807 10:51:16.229300 17868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:51:16.234079 17972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:51:16.238955 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 17868
I20260807 10:51:16.239234 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:51:16.239967 3367 external_mini_cluster.cc:1428] Setting key 02f345cf94db34d5e6e5fa9b95fd5d0c
I20260807 10:51:16.246050 17972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bc08c05f5b345acbec30b749cf41afd: Bootstrap starting.
I20260807 10:51:16.249462 17975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:16.242408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38120 (local address 127.3.73.254:34105)
0807 10:51:16.243601 (+ 1193us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:16.243614 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:16.243669 (+ 55us) server_negotiation.cc:407] Connection header received
0807 10:51:16.245455 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:16.245481 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:16.245931 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:16.246402 (+ 471us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:16.247626 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:51:16.247632 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:51:16.247646 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:51:16.247661 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:51:16.247737 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:51:16.247755 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:51:16.247936 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 10:51:16.248107 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 10:51:16.248426 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":682,"thread_start_us":452,"threads_started":1}
I20260807 10:51:16.250113 17972 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6bc08c05f5b345acbec30b749cf41afd: Neither blocks nor log segments found. Creating new log.
I20260807 10:51:16.251256 17972 log.cc:824] T 00000000000000000000000000000000 P 6bc08c05f5b345acbec30b749cf41afd: Log is configured to *not* fsync() on all Append() calls
I20260807 10:51:16.254566 17972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bc08c05f5b345acbec30b749cf41afd: No bootstrap required, opened a new log
I20260807 10:51:16.262774 17972 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6bc08c05f5b345acbec30b749cf41afd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bc08c05f5b345acbec30b749cf41afd" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 34105 } }
I20260807 10:51:16.263236 17972 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6bc08c05f5b345acbec30b749cf41afd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:51:16.263367 17972 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6bc08c05f5b345acbec30b749cf41afd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bc08c05f5b345acbec30b749cf41afd, State: Initialized, Role: FOLLOWER
I20260807 10:51:16.264036 17972 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6bc08c05f5b345acbec30b749cf41afd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bc08c05f5b345acbec30b749cf41afd" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 34105 } }
I20260807 10:51:16.264267 17972 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6bc08c05f5b345acbec30b749cf41afd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:51:16.264446 17972 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bc08c05f5b345acbec30b749cf41afd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:51:16.264634 17972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bc08c05f5b345acbec30b749cf41afd [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:51:16.267241 17972 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bc08c05f5b345acbec30b749cf41afd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bc08c05f5b345acbec30b749cf41afd" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 34105 } }
I20260807 10:51:16.267860 17972 leader_election.cc:302] T 00000000000000000000000000000000 P 6bc08c05f5b345acbec30b749cf41afd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6bc08c05f5b345acbec30b749cf41afd; no voters:
I20260807 10:51:16.268612 17972 leader_election.cc:288] T 00000000000000000000000000000000 P 6bc08c05f5b345acbec30b749cf41afd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:51:16.268841 17977 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bc08c05f5b345acbec30b749cf41afd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:51:16.269654 17977 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6bc08c05f5b345acbec30b749cf41afd [term 1 LEADER]: Becoming Leader. State: Replica: 6bc08c05f5b345acbec30b749cf41afd, State: Running, Role: LEADER
I20260807 10:51:16.270370 17977 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6bc08c05f5b345acbec30b749cf41afd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bc08c05f5b345acbec30b749cf41afd" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 34105 } }
I20260807 10:51:16.270973 17972 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6bc08c05f5b345acbec30b749cf41afd [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:51:16.273972 17979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bc08c05f5b345acbec30b749cf41afd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bc08c05f5b345acbec30b749cf41afd. Latest consensus state: current_term: 1 leader_uuid: "6bc08c05f5b345acbec30b749cf41afd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bc08c05f5b345acbec30b749cf41afd" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 34105 } } }
I20260807 10:51:16.274595 17979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bc08c05f5b345acbec30b749cf41afd [sys.catalog]: This master's current role is: LEADER
I20260807 10:51:16.274230 17978 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bc08c05f5b345acbec30b749cf41afd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6bc08c05f5b345acbec30b749cf41afd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bc08c05f5b345acbec30b749cf41afd" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 34105 } } }
I20260807 10:51:16.274947 17978 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bc08c05f5b345acbec30b749cf41afd [sys.catalog]: This master's current role is: LEADER
I20260807 10:51:16.276207 17992 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:51:16.281810 17992 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:51:16.291669 17992 catalog_manager.cc:1347] Generated new cluster ID: b44d6b859c234824b9106a810b918ae8
I20260807 10:51:16.291802 17992 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:51:16.305545 17992 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:51:16.306209 17992 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:51:16.313589 17992 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6bc08c05f5b345acbec30b749cf41afd: Generated new TSK 0
I20260807 10:51:16.314597 17992 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 10:51:16.330072 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.193:0
--local_ip_for_outbound_sockets=127.3.73.193
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:34105
--builtin_ntp_servers=127.3.73.212:35125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 10:51:16.707087 18003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:51:16.707389 18003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:51:16.707563 18003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:51:16.715389 18003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:51:16.715508 18003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:51:16.715688 18003 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.193
I20260807 10:51:16.725347 18003 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:34105
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.193
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:51:16.726747 18003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:51:16.728806 18003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:51:16.743309 18010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:51:16.743309 18011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:51:16.743309 18013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:51:16.744212 18003 server_base.cc:1034] running on GCE node
I20260807 10:51:16.745007 18003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:51:16.746096 18003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:51:16.747354 18003 hybrid_clock.cc:648] HybridClock initialized: now 1786099876747300 us; error 50 us; skew 500 ppm
I20260807 10:51:16.747714 18003 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 10:51:16.750481 18003 webserver.cc:460] Webserver started at http://127.3.73.193:38773/ using document root <none> and password file <none>
I20260807 10:51:16.751282 18003 fs_manager.cc:359] Metadata directory not provided
I20260807 10:51:16.751439 18003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:51:16.751825 18003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:51:16.755141 18003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/data/instance:
uuid: "72a4d711adcb493fa0bba5741343dec0"
format_stamp: "Formatted at 2026-08-07 10:51:16 on dist-test-slave-x5fp"
server_key: "2c0394e03e78856ae596d2dc92b56e78"
server_key_iv: "ce24c8db8f352bfb1839cf0eef73f0c9"
server_key_version: "encryptionkey@0"
I20260807 10:51:16.756047 18003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance:
uuid: "72a4d711adcb493fa0bba5741343dec0"
format_stamp: "Formatted at 2026-08-07 10:51:16 on dist-test-slave-x5fp"
server_key: "2c0394e03e78856ae596d2dc92b56e78"
server_key_iv: "ce24c8db8f352bfb1839cf0eef73f0c9"
server_key_version: "encryptionkey@0"
I20260807 10:51:16.761703 18003 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260807 10:51:16.765707 18024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:51:16.767194 18003 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 10:51:16.767403 18003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/wal
uuid: "72a4d711adcb493fa0bba5741343dec0"
format_stamp: "Formatted at 2026-08-07 10:51:16 on dist-test-slave-x5fp"
server_key: "2c0394e03e78856ae596d2dc92b56e78"
server_key_iv: "ce24c8db8f352bfb1839cf0eef73f0c9"
server_key_version: "encryptionkey@0"
I20260807 10:51:16.767661 18003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:51:16.774344 18003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:51:16.775377 18003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:51:16.775739 18003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:51:16.776896 18003 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:51:16.778492 18003 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:51:16.778625 18003 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:51:16.778777 18003 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:51:16.778872 18003 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:51:16.817355 18003 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.193:42159
I20260807 10:51:16.817421 18196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.193:42159 every 8 connection(s)
I20260807 10:51:16.818861 18003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
I20260807 10:51:16.826897 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 18003
I20260807 10:51:16.827142 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance
I20260807 10:51:16.827855 3367 external_mini_cluster.cc:1428] Setting key 0629beca1452af40cfbcf8f6b89f4452
I20260807 10:51:16.827853 18199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:16.821684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:46157 (local address 127.3.73.254:34105)
0807 10:51:16.822188 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:16.822192 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:16.823419 (+ 1227us) server_negotiation.cc:407] Connection header received
0807 10:51:16.825276 (+ 1857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:16.825283 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:16.825371 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:16.825496 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:16.826957 (+ 1461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:51:16.826986 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:51:16.826993 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:51:16.827019 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:51:16.827081 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:51:16.827096 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:51:16.827240 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 10:51:16.827444 (+ 204us) server_negotiation.cc:299] Negotiation successful
0807 10:51:16.827518 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":265,"threads_started":1}
I20260807 10:51:16.829203 18200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:16.822155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34105 (local address 127.3.73.193:46157)
0807 10:51:16.823232 (+ 1077us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:16.823278 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:16.824998 (+ 1720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:16.825741 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:16.825749 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:16.825848 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:16.826583 (+ 735us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 10:51:16.826585 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:16.826623 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 10:51:16.826710 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:51:16.827200 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:51:16.827214 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:51:16.827223 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 10:51:16.827533 (+ 310us) client_negotiation.cc:240] Negotiation successful
0807 10:51:16.827798 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":379,"threads_started":1}
I20260807 10:51:16.830861 18197 heartbeater.cc:347] Connected to a master server at 127.3.73.254:34105
I20260807 10:51:16.831032 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.194:0
--local_ip_for_outbound_sockets=127.3.73.194
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:34105
--builtin_ntp_servers=127.3.73.212:35125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 10:51:16.831261 18197 heartbeater.cc:464] Registering TS with master...
I20260807 10:51:16.831892 18197 heartbeater.cc:511] Master 127.3.73.254:34105 requested a full tablet report, sending...
I20260807 10:51:16.834142 17921 ts_manager.cc:194] Registered new tserver with Master: 72a4d711adcb493fa0bba5741343dec0 (127.3.73.193:42159)
W20260807 10:51:17.194590 18202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:51:17.194921 18202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:51:17.195019 18202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:51:17.202454 18202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:51:17.202587 18202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:51:17.202705 18202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.194
I20260807 10:51:17.213253 18202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:34105
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.194
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:51:17.214636 18202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:51:17.216461 18202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:51:17.232486 18210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:51:17.232491 18211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:51:17.233249 18216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:51:17.233806 18202 server_base.cc:1034] running on GCE node
I20260807 10:51:17.234613 18202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:51:17.235999 18202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:51:17.237365 18202 hybrid_clock.cc:648] HybridClock initialized: now 1786099877237240 us; error 118 us; skew 500 ppm
I20260807 10:51:17.237736 18202 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 10:51:17.240376 18202 webserver.cc:460] Webserver started at http://127.3.73.194:35473/ using document root <none> and password file <none>
I20260807 10:51:17.241232 18202 fs_manager.cc:359] Metadata directory not provided
I20260807 10:51:17.241384 18202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:51:17.241731 18202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:51:17.244796 18202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/data/instance:
uuid: "5f05e6437fb34df4a4136261fa5e1005"
format_stamp: "Formatted at 2026-08-07 10:51:17 on dist-test-slave-x5fp"
server_key: "ec35c2caafae04dbd1e50383d99a014c"
server_key_iv: "75e4e4730427e4e0ac21665062b1dfee"
server_key_version: "encryptionkey@0"
I20260807 10:51:17.245666 18202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance:
uuid: "5f05e6437fb34df4a4136261fa5e1005"
format_stamp: "Formatted at 2026-08-07 10:51:17 on dist-test-slave-x5fp"
server_key: "ec35c2caafae04dbd1e50383d99a014c"
server_key_iv: "75e4e4730427e4e0ac21665062b1dfee"
server_key_version: "encryptionkey@0"
I20260807 10:51:17.251230 18202 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 10:51:17.255033 18229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:51:17.256441 18202 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 10:51:17.256646 18202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/wal
uuid: "5f05e6437fb34df4a4136261fa5e1005"
format_stamp: "Formatted at 2026-08-07 10:51:17 on dist-test-slave-x5fp"
server_key: "ec35c2caafae04dbd1e50383d99a014c"
server_key_iv: "75e4e4730427e4e0ac21665062b1dfee"
server_key_version: "encryptionkey@0"
I20260807 10:51:17.256997 18202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:51:17.264096 18202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:51:17.265178 18202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:51:17.265470 18202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:51:17.266430 18202 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:51:17.267892 18202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:51:17.268024 18202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:51:17.268153 18202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:51:17.268242 18202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:51:17.302341 18202 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.194:38293
I20260807 10:51:17.302404 18387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.194:38293 every 8 connection(s)
I20260807 10:51:17.303865 18202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
I20260807 10:51:17.305280 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 18202
I20260807 10:51:17.305485 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance
I20260807 10:51:17.305950 3367 external_mini_cluster.cc:1428] Setting key c61fe8e085842ef1fbcf29a9f3b02b66
I20260807 10:51:17.309394 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.195:0
--local_ip_for_outbound_sockets=127.3.73.195
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:34105
--builtin_ntp_servers=127.3.73.212:35125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 10:51:17.314240 18199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:17.307848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:57355 (local address 127.3.73.254:34105)
0807 10:51:17.308054 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:17.308061 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:17.308687 (+ 626us) server_negotiation.cc:407] Connection header received
0807 10:51:17.311486 (+ 2799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:17.311490 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:17.311547 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:17.311637 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:17.313304 (+ 1667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:51:17.313314 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:51:17.313317 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:51:17.313334 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:51:17.313371 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:51:17.313383 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:51:17.313497 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 10:51:17.313782 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 10:51:17.313861 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 10:51:17.316161 18391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:17.307228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34105 (local address 127.3.73.194:57355)
0807 10:51:17.308506 (+ 1278us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:17.308560 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:17.311153 (+ 2593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:17.311924 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:17.311939 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:17.312064 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:17.312875 (+ 811us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 10:51:17.312881 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:17.312936 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 10:51:17.313102 (+ 166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:51:17.313492 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:51:17.313505 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:51:17.313516 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 10:51:17.314581 (+ 1065us) client_negotiation.cc:240] Negotiation successful
0807 10:51:17.314867 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":698,"thread_start_us":483,"threads_started":1}
I20260807 10:51:17.317884 18388 heartbeater.cc:347] Connected to a master server at 127.3.73.254:34105
I20260807 10:51:17.318222 18388 heartbeater.cc:464] Registering TS with master...
I20260807 10:51:17.318876 18388 heartbeater.cc:511] Master 127.3.73.254:34105 requested a full tablet report, sending...
I20260807 10:51:17.320395 17921 ts_manager.cc:194] Registered new tserver with Master: 5f05e6437fb34df4a4136261fa5e1005 (127.3.73.194:38293)
W20260807 10:51:17.672987 18392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:51:17.673295 18392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:51:17.673451 18392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:51:17.681025 18392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:51:17.681180 18392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:51:17.681376 18392 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.195
I20260807 10:51:17.691015 18392 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:34105
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.195
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:51:17.692438 18392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:51:17.694345 18392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:51:17.708935 18400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:51:17.708901 18401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:51:17.708959 18404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:51:17.709599 18392 server_base.cc:1034] running on GCE node
I20260807 10:51:17.710364 18392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:51:17.711457 18392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:51:17.712710 18392 hybrid_clock.cc:648] HybridClock initialized: now 1786099877712669 us; error 62 us; skew 500 ppm
I20260807 10:51:17.713140 18392 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 10:51:17.715684 18392 webserver.cc:460] Webserver started at http://127.3.73.195:46723/ using document root <none> and password file <none>
I20260807 10:51:17.716462 18392 fs_manager.cc:359] Metadata directory not provided
I20260807 10:51:17.716614 18392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:51:17.717046 18392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:51:17.720237 18392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/data/instance:
uuid: "ed1609df68c4426bb546a75b9a750df7"
format_stamp: "Formatted at 2026-08-07 10:51:17 on dist-test-slave-x5fp"
server_key: "43d2fbd0568157eeddaa72bfda5bfd16"
server_key_iv: "714816345ceb3fb94953c75038f84e6b"
server_key_version: "encryptionkey@0"
I20260807 10:51:17.721205 18392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance:
uuid: "ed1609df68c4426bb546a75b9a750df7"
format_stamp: "Formatted at 2026-08-07 10:51:17 on dist-test-slave-x5fp"
server_key: "43d2fbd0568157eeddaa72bfda5bfd16"
server_key_iv: "714816345ceb3fb94953c75038f84e6b"
server_key_version: "encryptionkey@0"
I20260807 10:51:17.726997 18392 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 10:51:17.730911 18414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:51:17.732530 18392 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 10:51:17.732820 18392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/wal
uuid: "ed1609df68c4426bb546a75b9a750df7"
format_stamp: "Formatted at 2026-08-07 10:51:17 on dist-test-slave-x5fp"
server_key: "43d2fbd0568157eeddaa72bfda5bfd16"
server_key_iv: "714816345ceb3fb94953c75038f84e6b"
server_key_version: "encryptionkey@0"
I20260807 10:51:17.733142 18392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:51:17.739212 18392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:51:17.740108 18392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:51:17.740397 18392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:51:17.741454 18392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:51:17.742939 18392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:51:17.743064 18392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:51:17.743227 18392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:51:17.743319 18392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:51:17.777971 18392 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.195:40057
I20260807 10:51:17.778028 18586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.195:40057 every 8 connection(s)
I20260807 10:51:17.779325 18392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
I20260807 10:51:17.784584 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 18392
I20260807 10:51:17.784821 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance
I20260807 10:51:17.785269 3367 external_mini_cluster.cc:1428] Setting key 69f8d1fa7cab7dc4f7805895f071d73c
I20260807 10:51:17.788066 18199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:17.782091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:57199 (local address 127.3.73.254:34105)
0807 10:51:17.782286 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:17.782291 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:17.783783 (+ 1492us) server_negotiation.cc:407] Connection header received
0807 10:51:17.785756 (+ 1973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:17.785762 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:17.785838 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:17.785952 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:17.787300 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:51:17.787306 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:51:17.787311 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:51:17.787331 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:51:17.787377 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:51:17.787389 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:51:17.787504 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 10:51:17.787714 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 10:51:17.787790 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 10:51:17.789106 18590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:17.782651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34105 (local address 127.3.73.195:57199)
0807 10:51:17.783626 (+ 975us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:17.783667 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:17.785478 (+ 1811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:17.786198 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:17.786206 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:17.786299 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:17.787003 (+ 704us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 10:51:17.787005 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:17.787042 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 10:51:17.787127 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:51:17.787494 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:51:17.787505 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:51:17.787513 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 10:51:17.787801 (+ 288us) client_negotiation.cc:240] Negotiation successful
0807 10:51:17.788068 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":341,"threads_started":1}
I20260807 10:51:17.790383 18588 heartbeater.cc:347] Connected to a master server at 127.3.73.254:34105
I20260807 10:51:17.790685 18588 heartbeater.cc:464] Registering TS with master...
I20260807 10:51:17.791267 18588 heartbeater.cc:511] Master 127.3.73.254:34105 requested a full tablet report, sending...
I20260807 10:51:17.792659 17921 ts_manager.cc:194] Registered new tserver with Master: ed1609df68c4426bb546a75b9a750df7 (127.3.73.195:40057)
I20260807 10:51:17.800571 3367 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 10:51:17.806881 18199 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:51:17.805259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38136 (local address 127.3.73.254:34105)
0807 10:51:17.805489 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:17.805497 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:17.806006 (+ 509us) server_negotiation.cc:407] Connection header received
0807 10:51:17.806147 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:17.806151 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:17.806204 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:17.806284 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:17.806631 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38136: BlockingRecv error: recv got EOF from 127.0.0.1:38136 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 10:51:17.807955 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 18003
I20260807 10:51:17.811684 18598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:17.808977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58758 (local address 127.3.73.193:42159)
0807 10:51:17.809479 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:17.809485 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:17.809510 (+ 25us) server_negotiation.cc:407] Connection header received
0807 10:51:17.809569 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:17.809574 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:17.809771 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:17.809951 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:17.810967 (+ 1016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:51:17.810973 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:51:17.810984 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:51:17.811003 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:51:17.811043 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:51:17.811057 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:51:17.811201 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 10:51:17.811307 (+ 106us) server_negotiation.cc:299] Negotiation successful
0807 10:51:17.811407 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":236,"threads_started":1}
I20260807 10:51:17.813064 18191 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:51:17.896934 18197 heartbeater.cc:503] Master 127.3.73.254:34105 was elected leader, sending a full tablet report...
I20260807 10:51:17.898274 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 18003
W20260807 10:51:17.925050 17901 connection.cc:535] server connection from 127.3.73.193:46157 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 10:51:17.925355 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 18202
I20260807 10:51:17.929608 18602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:17.926539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52146 (local address 127.3.73.194:38293)
0807 10:51:17.927102 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:17.927109 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:17.927141 (+ 32us) server_negotiation.cc:407] Connection header received
0807 10:51:17.927211 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:17.927228 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:17.927485 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:17.927627 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:17.928710 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:51:17.928780 (+ 70us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:51:17.928800 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:51:17.928827 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:51:17.928890 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:51:17.928914 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:51:17.929045 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 10:51:17.929168 (+ 123us) server_negotiation.cc:299] Negotiation successful
0807 10:51:17.929318 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":264,"threads_started":1}
I20260807 10:51:17.930862 18383 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:51:18.023979 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 18202
I20260807 10:51:18.050592 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 18392
I20260807 10:51:18.055269 18604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:18.051751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48524 (local address 127.3.73.195:40057)
0807 10:51:18.052525 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:18.052533 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:18.052570 (+ 37us) server_negotiation.cc:407] Connection header received
0807 10:51:18.052643 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:18.052649 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:18.052994 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:18.053141 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:18.054205 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:51:18.054213 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:51:18.054225 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:51:18.054269 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:51:18.054319 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:51:18.054335 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:51:18.054490 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 10:51:18.054725 (+ 235us) server_negotiation.cc:299] Negotiation successful
0807 10:51:18.054886 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":385,"threads_started":1}
I20260807 10:51:18.056701 18575 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:51:18.142370 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 18392
I20260807 10:51:18.166554 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 17868
I20260807 10:51:18.168000 17965 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:51:18.277278 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 17868
2026-08-07T10:51:18Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2602 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:51:18 dist-test-slave-x5fp krb5kdc[18609](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:51:18 dist-test-slave-x5fp krb5kdc[18609](info): set up 2 sockets
Aug 07 10:51:18 dist-test-slave-x5fp krb5kdc[18609](info): commencing operation
krb5kdc: starting...
W20260807 10:51:19.634812 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.307s user 0.002s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:51:19 dist-test-slave-x5fp krb5kdc[18609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:51:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:51:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.73.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
I20260807 10:51:19.985438 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:42229
--webserver_interface=127.3.73.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:33311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:42229
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:51:20.343580 18627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:51:20.343978 18627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:51:20.344151 18627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:51:20.351611 18627 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:51:20.351797 18627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:51:20.351919 18627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:51:20.352028 18627 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:51:20.352108 18627 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:51:20.362530 18627 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:33311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:42229
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:42229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:51:20.364059 18627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:51:20.365557 18627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:51:20.378597 18634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:51:20.378638 18640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:51:20.378660 18638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:51:20.379576 18627 server_base.cc:1034] running on GCE node
I20260807 10:51:20.380496 18627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:51:20.382119 18627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:51:20.383394 18627 hybrid_clock.cc:648] HybridClock initialized: now 1786099880383318 us; error 69 us; skew 500 ppm
I20260807 10:51:20.383937 18627 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:51:20 dist-test-slave-x5fp krb5kdc[18609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099880, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:51:20.387759 18627 init.cc:377] Logged in from keytab as kudu/127.3.73.254@KRBTEST.COM (short username kudu)
I20260807 10:51:20.389652 18627 webserver.cc:460] Webserver started at http://127.3.73.254:43857/ using document root <none> and password file <none>
I20260807 10:51:20.390401 18627 fs_manager.cc:359] Metadata directory not provided
I20260807 10:51:20.390560 18627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:51:20.390892 18627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:51:20.394160 18627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "597634295a16444ab7c7f5acc99bd3db"
format_stamp: "Formatted at 2026-08-07 10:51:20 on dist-test-slave-x5fp"
server_key: "bcc2c9e8e126f7f33978d29aaa7f57bd"
server_key_iv: "b48aace27c52c600fc15e3779ca1d49b"
server_key_version: "encryptionkey@0"
I20260807 10:51:20.394982 18627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "597634295a16444ab7c7f5acc99bd3db"
format_stamp: "Formatted at 2026-08-07 10:51:20 on dist-test-slave-x5fp"
server_key: "bcc2c9e8e126f7f33978d29aaa7f57bd"
server_key_iv: "b48aace27c52c600fc15e3779ca1d49b"
server_key_version: "encryptionkey@0"
I20260807 10:51:20.400509 18627 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 10:51:20.404143 18649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:51:20.405611 18627 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 10:51:20.405820 18627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "597634295a16444ab7c7f5acc99bd3db"
format_stamp: "Formatted at 2026-08-07 10:51:20 on dist-test-slave-x5fp"
server_key: "bcc2c9e8e126f7f33978d29aaa7f57bd"
server_key_iv: "b48aace27c52c600fc15e3779ca1d49b"
server_key_version: "encryptionkey@0"
I20260807 10:51:20.406078 18627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:51:20.424894 18627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:51:20.432467 18627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:51:20.432811 18627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:51:20.453250 18627 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:42229
I20260807 10:51:20.453274 18735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:42229 every 8 connection(s)
I20260807 10:51:20.454768 18627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:51:20.458853 18736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:51:20.460525 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 18627
I20260807 10:51:20.460825 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:51:20.461681 3367 external_mini_cluster.cc:1428] Setting key 96e8e3c2cb0cddd91352f8b080557d97
I20260807 10:51:20.472467 18736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 597634295a16444ab7c7f5acc99bd3db: Bootstrap starting.
I20260807 10:51:20.475862 18736 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 597634295a16444ab7c7f5acc99bd3db: Neither blocks nor log segments found. Creating new log.
Aug 07 10:51:20 dist-test-slave-x5fp krb5kdc[18609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:51:20.477277 18736 log.cc:824] T 00000000000000000000000000000000 P 597634295a16444ab7c7f5acc99bd3db: Log is configured to *not* fsync() on all Append() calls
I20260807 10:51:20.480623 18736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 597634295a16444ab7c7f5acc99bd3db: No bootstrap required, opened a new log
I20260807 10:51:20.486287 18739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:20.464050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47176 (local address 127.3.73.254:42229)
0807 10:51:20.464987 (+ 937us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:20.464999 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:20.465052 (+ 53us) server_negotiation.cc:407] Connection header received
0807 10:51:20.466647 (+ 1595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:20.466674 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:20.467076 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:20.467468 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:20.469082 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:20.470447 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:20.471748 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:20.472445 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:20.472642 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:20.477700 (+ 5058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:51:20.477711 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:51:20.477728 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:51:20.477770 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:51:20.481483 (+ 3713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:51:20.482291 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:51:20.482298 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:51:20.482305 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:51:20.482493 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:51:20.482939 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:51:20.482945 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:51:20.482949 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:51:20.483605 (+ 656us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:51:20.484027 (+ 422us) server_negotiation.cc:1035] Waiting for connection context
0807 10:51:20.484484 (+ 457us) server_negotiation.cc:299] Negotiation successful
0807 10:51:20.485109 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":380,"threads_started":1}
I20260807 10:51:20.489599 18736 raft_consensus.cc:348] T 00000000000000000000000000000000 P 597634295a16444ab7c7f5acc99bd3db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "597634295a16444ab7c7f5acc99bd3db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 42229 } }
I20260807 10:51:20.490093 18736 raft_consensus.cc:374] T 00000000000000000000000000000000 P 597634295a16444ab7c7f5acc99bd3db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:51:20.490207 18736 raft_consensus.cc:729] T 00000000000000000000000000000000 P 597634295a16444ab7c7f5acc99bd3db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 597634295a16444ab7c7f5acc99bd3db, State: Initialized, Role: FOLLOWER
I20260807 10:51:20.490790 18736 consensus_queue.cc:260] T 00000000000000000000000000000000 P 597634295a16444ab7c7f5acc99bd3db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "597634295a16444ab7c7f5acc99bd3db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 42229 } }
I20260807 10:51:20.491029 18736 raft_consensus.cc:388] T 00000000000000000000000000000000 P 597634295a16444ab7c7f5acc99bd3db [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:51:20.491185 18736 raft_consensus.cc:482] T 00000000000000000000000000000000 P 597634295a16444ab7c7f5acc99bd3db [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:51:20.491374 18736 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 597634295a16444ab7c7f5acc99bd3db [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:51:20.493839 18736 raft_consensus.cc:504] T 00000000000000000000000000000000 P 597634295a16444ab7c7f5acc99bd3db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "597634295a16444ab7c7f5acc99bd3db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 42229 } }
I20260807 10:51:20.494382 18736 leader_election.cc:302] T 00000000000000000000000000000000 P 597634295a16444ab7c7f5acc99bd3db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 597634295a16444ab7c7f5acc99bd3db; no voters:
I20260807 10:51:20.495014 18736 leader_election.cc:288] T 00000000000000000000000000000000 P 597634295a16444ab7c7f5acc99bd3db [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:51:20.495200 18743 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 597634295a16444ab7c7f5acc99bd3db [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:51:20.496671 18743 raft_consensus.cc:686] T 00000000000000000000000000000000 P 597634295a16444ab7c7f5acc99bd3db [term 1 LEADER]: Becoming Leader. State: Replica: 597634295a16444ab7c7f5acc99bd3db, State: Running, Role: LEADER
I20260807 10:51:20.497217 18743 consensus_queue.cc:237] T 00000000000000000000000000000000 P 597634295a16444ab7c7f5acc99bd3db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "597634295a16444ab7c7f5acc99bd3db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 42229 } }
I20260807 10:51:20.497478 18736 sys_catalog.cc:564] T 00000000000000000000000000000000 P 597634295a16444ab7c7f5acc99bd3db [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:51:20.500373 18745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 597634295a16444ab7c7f5acc99bd3db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 597634295a16444ab7c7f5acc99bd3db. Latest consensus state: current_term: 1 leader_uuid: "597634295a16444ab7c7f5acc99bd3db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "597634295a16444ab7c7f5acc99bd3db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 42229 } } }
I20260807 10:51:20.500748 18745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 597634295a16444ab7c7f5acc99bd3db [sys.catalog]: This master's current role is: LEADER
I20260807 10:51:20.501391 18744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 597634295a16444ab7c7f5acc99bd3db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "597634295a16444ab7c7f5acc99bd3db" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "597634295a16444ab7c7f5acc99bd3db" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 42229 } } }
I20260807 10:51:20.501739 18744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 597634295a16444ab7c7f5acc99bd3db [sys.catalog]: This master's current role is: LEADER
I20260807 10:51:20.504813 18760 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:51:20.510252 18760 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:51:20.520226 18760 catalog_manager.cc:1347] Generated new cluster ID: 4af41dd9198c44f494f7ffeb8b892154
I20260807 10:51:20.520381 18760 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:51:20.537312 18760 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:51:20.538703 18760 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:51:20.550015 18760 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 597634295a16444ab7c7f5acc99bd3db: Generated new TSK 0
I20260807 10:51:20.551188 18760 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.73.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab.
Entry for principal kudu/127.3.73.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab.
I20260807 10:51:20.633802 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.193:0
--local_ip_for_outbound_sockets=127.3.73.193
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:42229
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:33311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:51:20.988837 18779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:51:20.989332 18779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:51:20.989514 18779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:51:20.997151 18779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:51:20.997263 18779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:51:20.997454 18779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.193
I20260807 10:51:21.007992 18779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:33311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:42229
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.193
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:51:21.009425 18779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:51:21.010875 18779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:51:21.024791 18794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:51:21.024921 18792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:51:21.025166 18790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:51:21.025712 18779 server_base.cc:1034] running on GCE node
I20260807 10:51:21.026310 18779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:51:21.027346 18779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:51:21.028563 18779 hybrid_clock.cc:648] HybridClock initialized: now 1786099881028520 us; error 32 us; skew 500 ppm
I20260807 10:51:21.028926 18779 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:51:21 dist-test-slave-x5fp krb5kdc[18609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099881, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:51:21.032380 18779 init.cc:377] Logged in from keytab as kudu/127.3.73.193@KRBTEST.COM (short username kudu)
I20260807 10:51:21.034075 18779 webserver.cc:460] Webserver started at http://127.3.73.193:44719/ using document root <none> and password file <none>
I20260807 10:51:21.034819 18779 fs_manager.cc:359] Metadata directory not provided
I20260807 10:51:21.034937 18779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:51:21.035207 18779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:51:21.038350 18779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/data/instance:
uuid: "decdd2305ec143f683ca4743b8a63087"
format_stamp: "Formatted at 2026-08-07 10:51:21 on dist-test-slave-x5fp"
server_key: "776bc0b5fad6243b579d39e8cbbf9efc"
server_key_iv: "e7d9ef99011a5fba4a2d1e808b6a3922"
server_key_version: "encryptionkey@0"
I20260807 10:51:21.039140 18779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance:
uuid: "decdd2305ec143f683ca4743b8a63087"
format_stamp: "Formatted at 2026-08-07 10:51:21 on dist-test-slave-x5fp"
server_key: "776bc0b5fad6243b579d39e8cbbf9efc"
server_key_iv: "e7d9ef99011a5fba4a2d1e808b6a3922"
server_key_version: "encryptionkey@0"
I20260807 10:51:21.044945 18779 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 10:51:21.048574 18808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:51:21.050290 18779 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 10:51:21.050484 18779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/wal
uuid: "decdd2305ec143f683ca4743b8a63087"
format_stamp: "Formatted at 2026-08-07 10:51:21 on dist-test-slave-x5fp"
server_key: "776bc0b5fad6243b579d39e8cbbf9efc"
server_key_iv: "e7d9ef99011a5fba4a2d1e808b6a3922"
server_key_version: "encryptionkey@0"
I20260807 10:51:21.050726 18779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:51:21.069797 18779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:51:21.074558 18779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:51:21.074918 18779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:51:21.076068 18779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:51:21.077625 18779 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:51:21.077702 18779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:51:21.077864 18779 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:51:21.077932 18779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:51:21.114322 18779 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.193:37353
I20260807 10:51:21.114456 18974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.193:37353 every 8 connection(s)
I20260807 10:51:21.115716 18779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
I20260807 10:51:21.116232 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 18779
I20260807 10:51:21.116503 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance
I20260807 10:51:21.117511 3367 external_mini_cluster.cc:1428] Setting key 5d41ea9fd0fc0e117db713c2e195b4d6
Aug 07 10:51:21 dist-test-slave-x5fp krb5kdc[18609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099881, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:51:21.139343 18978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:21.118786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:45359 (local address 127.3.73.254:42229)
0807 10:51:21.119294 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:21.119300 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:21.120395 (+ 1095us) server_negotiation.cc:407] Connection header received
0807 10:51:21.122318 (+ 1923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:21.122323 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:21.122419 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:21.122542 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:21.124101 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:21.124910 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:21.126882 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:21.127680 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:21.127847 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:21.132771 (+ 4924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:51:21.132779 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:51:21.132787 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:51:21.132833 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:51:21.135877 (+ 3044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:51:21.136768 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:51:21.136776 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:51:21.136781 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:51:21.136917 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:51:21.137406 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:51:21.137417 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:51:21.137420 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:51:21.137757 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:51:21.137933 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 10:51:21.138705 (+ 772us) server_negotiation.cc:299] Negotiation successful
0807 10:51:21.138952 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":258,"threads_started":1}
I20260807 10:51:21.140415 18979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:21.119418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:42229 (local address 127.3.73.193:45359)
0807 10:51:21.120245 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:21.120285 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:21.122041 (+ 1756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:21.122790 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:21.122799 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:21.123331 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:21.123918 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:21.123936 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:21.125081 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:21.125086 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:21.126694 (+ 1608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:21.126713 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:21.127933 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:21.127941 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:21.128251 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:21.129121 (+ 870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:21.129160 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:21.132474 (+ 3314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:51:21.136047 (+ 3573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:51:21.136054 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:51:21.136068 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:51:21.136577 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:51:21.137046 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:51:21.137052 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:51:21.137055 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:51:21.137274 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:51:21.137921 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:51:21.137929 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:51:21.138302 (+ 373us) client_negotiation.cc:769] Sending connection context
0807 10:51:21.138597 (+ 295us) client_negotiation.cc:240] Negotiation successful
0807 10:51:21.139171 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":352,"threads_started":1}
I20260807 10:51:21.142198 18976 heartbeater.cc:347] Connected to a master server at 127.3.73.254:42229
I20260807 10:51:21.142556 18976 heartbeater.cc:464] Registering TS with master...
I20260807 10:51:21.143352 18976 heartbeater.cc:511] Master 127.3.73.254:42229 requested a full tablet report, sending...
I20260807 10:51:21.145967 18675 ts_manager.cc:194] Registered new tserver with Master: decdd2305ec143f683ca4743b8a63087 (127.3.73.193:37353)
I20260807 10:51:21.148329 18675 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.193@KRBTEST.COM'} at 127.3.73.193:45359
WARNING: no policy specified for kudu/127.3.73.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab.
Entry for principal kudu/127.3.73.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab.
I20260807 10:51:21.194867 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.194:0
--local_ip_for_outbound_sockets=127.3.73.194
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:42229
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:33311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:51:21.559161 18982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:51:21.559476 18982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:51:21.559574 18982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:51:21.566931 18982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:51:21.567063 18982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:51:21.567180 18982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.194
I20260807 10:51:21.577666 18982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:33311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:42229
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.194
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:51:21.579077 18982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:51:21.580423 18982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:51:21.594717 18993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:51:21.594756 18997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:51:21.594655 18990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:51:21.596329 18982 server_base.cc:1034] running on GCE node
I20260807 10:51:21.597146 18982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:51:21.598315 18982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:51:21.599588 18982 hybrid_clock.cc:648] HybridClock initialized: now 1786099881599513 us; error 59 us; skew 500 ppm
I20260807 10:51:21.599956 18982 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:51:21 dist-test-slave-x5fp krb5kdc[18609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099881, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:51:21.603503 18982 init.cc:377] Logged in from keytab as kudu/127.3.73.194@KRBTEST.COM (short username kudu)
I20260807 10:51:21.605374 18982 webserver.cc:460] Webserver started at http://127.3.73.194:43417/ using document root <none> and password file <none>
I20260807 10:51:21.606122 18982 fs_manager.cc:359] Metadata directory not provided
I20260807 10:51:21.606278 18982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:51:21.606608 18982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:51:21.609907 18982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/data/instance:
uuid: "ed15463bc70c42dfa8c753ac6b7dbbc9"
format_stamp: "Formatted at 2026-08-07 10:51:21 on dist-test-slave-x5fp"
server_key: "aab9608e9bf442bdb36bf4d420852694"
server_key_iv: "d250c4909fe08b755ee6bb04c7d96f51"
server_key_version: "encryptionkey@0"
I20260807 10:51:21.610929 18982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance:
uuid: "ed15463bc70c42dfa8c753ac6b7dbbc9"
format_stamp: "Formatted at 2026-08-07 10:51:21 on dist-test-slave-x5fp"
server_key: "aab9608e9bf442bdb36bf4d420852694"
server_key_iv: "d250c4909fe08b755ee6bb04c7d96f51"
server_key_version: "encryptionkey@0"
I20260807 10:51:21.616549 18982 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 10:51:21.620421 19012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:51:21.621920 18982 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 10:51:21.622126 18982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/wal
uuid: "ed15463bc70c42dfa8c753ac6b7dbbc9"
format_stamp: "Formatted at 2026-08-07 10:51:21 on dist-test-slave-x5fp"
server_key: "aab9608e9bf442bdb36bf4d420852694"
server_key_iv: "d250c4909fe08b755ee6bb04c7d96f51"
server_key_version: "encryptionkey@0"
I20260807 10:51:21.622355 18982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:51:21.652661 18982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:51:21.657527 18982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:51:21.657861 18982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:51:21.658905 18982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:51:21.660446 18982 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:51:21.660554 18982 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:51:21.660729 18982 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:51:21.660817 18982 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:51:21.696051 18982 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.194:33221
I20260807 10:51:21.696149 19179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.194:33221 every 8 connection(s)
I20260807 10:51:21.697479 18982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
I20260807 10:51:21.701207 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 18982
I20260807 10:51:21.701431 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance
I20260807 10:51:21.701982 3367 external_mini_cluster.cc:1428] Setting key 80934aa4b1de68979941defe0aaf0cbe
Aug 07 10:51:21 dist-test-slave-x5fp krb5kdc[18609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099881, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:51:21.721426 19185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:21.700209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:41047 (local address 127.3.73.254:42229)
0807 10:51:21.700774 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:21.700781 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:21.701846 (+ 1065us) server_negotiation.cc:407] Connection header received
0807 10:51:21.704187 (+ 2341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:21.704193 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:21.704280 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:21.704402 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:21.706099 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:21.707318 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:21.709007 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:21.709744 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:21.709889 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:21.713887 (+ 3998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:51:21.713896 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:51:21.713903 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:51:21.713950 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:51:21.717842 (+ 3892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:51:21.718737 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:51:21.718745 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:51:21.718751 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:51:21.718940 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:51:21.719527 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:51:21.719530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:51:21.719533 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:51:21.719807 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:51:21.719924 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 10:51:21.720946 (+ 1022us) server_negotiation.cc:299] Negotiation successful
0807 10:51:21.721128 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":231,"threads_started":1}
I20260807 10:51:21.723054 19187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:21.700787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:42229 (local address 127.3.73.194:41047)
0807 10:51:21.701641 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:21.701697 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:21.703829 (+ 2132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:21.704647 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:21.704656 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:21.705277 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:21.705927 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:21.705945 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:21.707502 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:21.707505 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:21.708861 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:21.708874 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:21.709986 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:21.709991 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:21.710292 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:21.711186 (+ 894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:21.711250 (+ 64us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:21.713590 (+ 2340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:51:21.718050 (+ 4460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:51:21.718058 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:51:21.718074 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:51:21.718584 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:51:21.719058 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:51:21.719061 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:51:21.719063 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:51:21.719374 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:51:21.719944 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:51:21.719954 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:51:21.720490 (+ 536us) client_negotiation.cc:769] Sending connection context
0807 10:51:21.720847 (+ 357us) client_negotiation.cc:240] Negotiation successful
0807 10:51:21.721493 (+ 646us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":353,"threads_started":1}
I20260807 10:51:21.724674 19180 heartbeater.cc:347] Connected to a master server at 127.3.73.254:42229
I20260807 10:51:21.725099 19180 heartbeater.cc:464] Registering TS with master...
I20260807 10:51:21.726008 19180 heartbeater.cc:511] Master 127.3.73.254:42229 requested a full tablet report, sending...
I20260807 10:51:21.727824 18675 ts_manager.cc:194] Registered new tserver with Master: ed15463bc70c42dfa8c753ac6b7dbbc9 (127.3.73.194:33221)
I20260807 10:51:21.728698 18675 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.194@KRBTEST.COM'} at 127.3.73.194:41047
WARNING: no policy specified for kudu/127.3.73.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab.
Entry for principal kudu/127.3.73.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab.
I20260807 10:51:21.781265 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.195:0
--local_ip_for_outbound_sockets=127.3.73.195
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:42229
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:33311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:51:22.151522 19190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
I20260807 10:51:22.151813 18976 heartbeater.cc:503] Master 127.3.73.254:42229 was elected leader, sending a full tablet report...
W20260807 10:51:22.151921 19190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:51:22.152122 19190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:51:22.160233 19190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:51:22.160435 19190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:51:22.160646 19190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.195
I20260807 10:51:22.171118 19190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:33311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:42229
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.195
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:51:22.172868 19190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:51:22.174311 19190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:51:22.189067 19198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:51:22.189069 19199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:51:22.189203 19202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:51:22.190090 19190 server_base.cc:1034] running on GCE node
I20260807 10:51:22.190760 19190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:51:22.191820 19190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:51:22.193094 19190 hybrid_clock.cc:648] HybridClock initialized: now 1786099882193046 us; error 41 us; skew 500 ppm
I20260807 10:51:22.193442 19190 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:51:22 dist-test-slave-x5fp krb5kdc[18609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099882, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:51:22.196679 19190 init.cc:377] Logged in from keytab as kudu/127.3.73.195@KRBTEST.COM (short username kudu)
I20260807 10:51:22.198390 19190 webserver.cc:460] Webserver started at http://127.3.73.195:46047/ using document root <none> and password file <none>
I20260807 10:51:22.199136 19190 fs_manager.cc:359] Metadata directory not provided
I20260807 10:51:22.199294 19190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:51:22.199609 19190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:51:22.202737 19190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/data/instance:
uuid: "eb2ca21f9547448d988820988f5355d8"
format_stamp: "Formatted at 2026-08-07 10:51:22 on dist-test-slave-x5fp"
server_key: "d682d316b798af3d7e09ce037ebb7171"
server_key_iv: "2eba33038acbd5ceafc6c548690fa80a"
server_key_version: "encryptionkey@0"
I20260807 10:51:22.203609 19190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance:
uuid: "eb2ca21f9547448d988820988f5355d8"
format_stamp: "Formatted at 2026-08-07 10:51:22 on dist-test-slave-x5fp"
server_key: "d682d316b798af3d7e09ce037ebb7171"
server_key_iv: "2eba33038acbd5ceafc6c548690fa80a"
server_key_version: "encryptionkey@0"
I20260807 10:51:22.209461 19190 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 10:51:22.213541 19213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:51:22.214977 19190 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 10:51:22.215173 19190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/wal
uuid: "eb2ca21f9547448d988820988f5355d8"
format_stamp: "Formatted at 2026-08-07 10:51:22 on dist-test-slave-x5fp"
server_key: "d682d316b798af3d7e09ce037ebb7171"
server_key_iv: "2eba33038acbd5ceafc6c548690fa80a"
server_key_version: "encryptionkey@0"
I20260807 10:51:22.215400 19190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:51:22.236056 19190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:51:22.240913 19190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:51:22.241227 19190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:51:22.242264 19190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:51:22.243664 19190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:51:22.243775 19190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:51:22.243943 19190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:51:22.244021 19190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:51:22.279659 19190 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.195:38223
I20260807 10:51:22.279789 19384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.195:38223 every 8 connection(s)
I20260807 10:51:22.281049 19190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
I20260807 10:51:22.286141 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 19190
I20260807 10:51:22.286355 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance
I20260807 10:51:22.286911 3367 external_mini_cluster.cc:1428] Setting key fca8f93c9db285175423e42954915b5b
Aug 07 10:51:22 dist-test-slave-x5fp krb5kdc[18609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099882, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:51:22.303452 19387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:22.283616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:47047 (local address 127.3.73.254:42229)
0807 10:51:22.284118 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:22.284125 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:22.285134 (+ 1009us) server_negotiation.cc:407] Connection header received
0807 10:51:22.287575 (+ 2441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:22.287581 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:22.287681 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:22.287834 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:22.289663 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:22.290608 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:22.292159 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:22.292562 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:22.292702 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:22.296317 (+ 3615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:51:22.296322 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:51:22.296325 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:51:22.296358 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:51:22.299680 (+ 3322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:51:22.300600 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:51:22.300604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:51:22.300606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:51:22.300812 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:51:22.301404 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:51:22.301408 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:51:22.301411 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:51:22.301775 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:51:22.302002 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 10:51:22.302982 (+ 980us) server_negotiation.cc:299] Negotiation successful
0807 10:51:22.303162 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":236,"threads_started":1}
I20260807 10:51:22.304638 19388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:22.284151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:42229 (local address 127.3.73.195:47047)
0807 10:51:22.284961 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:22.285004 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:22.287200 (+ 2196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:22.288097 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:22.288105 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:22.288892 (+ 787us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:51:22.289483 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:22.289501 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:22.290775 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:22.290779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:22.292002 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:22.292016 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:22.292832 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:22.292840 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:22.293169 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:22.293892 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:51:22.293922 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:51:22.296120 (+ 2198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:51:22.299842 (+ 3722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:51:22.299853 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:51:22.299871 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:51:22.300458 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:51:22.300968 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:51:22.300973 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:51:22.300976 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:51:22.301258 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:51:22.301943 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:51:22.301973 (+ 30us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:51:22.302461 (+ 488us) client_negotiation.cc:769] Sending connection context
0807 10:51:22.302848 (+ 387us) client_negotiation.cc:240] Negotiation successful
0807 10:51:22.303318 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":289,"threads_started":1}
I20260807 10:51:22.306171 19385 heartbeater.cc:347] Connected to a master server at 127.3.73.254:42229
I20260807 10:51:22.306516 19385 heartbeater.cc:464] Registering TS with master...
I20260807 10:51:22.307336 19385 heartbeater.cc:511] Master 127.3.73.254:42229 requested a full tablet report, sending...
I20260807 10:51:22.308919 18675 ts_manager.cc:194] Registered new tserver with Master: eb2ca21f9547448d988820988f5355d8 (127.3.73.195:38223)
I20260807 10:51:22.309705 18675 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.195@KRBTEST.COM'} at 127.3.73.195:47047
I20260807 10:51:22.315114 3367 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 10:51:22.333073 19387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:22.319983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47180 (local address 127.3.73.254:42229)
0807 10:51:22.320205 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:22.320212 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:22.320699 (+ 487us) server_negotiation.cc:407] Connection header received
0807 10:51:22.320876 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:22.320880 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:22.320946 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:22.321030 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:22.322525 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:22.323260 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:22.324486 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:22.325014 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:22.325113 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:22.327712 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:51:22.327717 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:51:22.327720 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:51:22.327754 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:51:22.330475 (+ 2721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:51:22.331101 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:51:22.331105 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:51:22.331107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:51:22.331227 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:51:22.331561 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:51:22.331564 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:51:22.331566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:51:22.331900 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:51:22.332085 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 10:51:22.332462 (+ 377us) server_negotiation.cc:299] Negotiation successful
0807 10:51:22.332692 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 10:51:22.337390 18675 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47180:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 10:51:22.344489 18675 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 10:51:22.371862 19402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:22.357645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58576 (local address 127.3.73.194:33221)
0807 10:51:22.358469 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:22.358480 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:22.359113 (+ 633us) server_negotiation.cc:407] Connection header received
0807 10:51:22.359564 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:22.359576 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:22.359897 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:22.360095 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:51:22.361366 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:22.363548 (+ 2182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:22.367575 (+ 4027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:22.369708 (+ 2133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:22.369866 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:22.370139 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0807 10:51:22.371086 (+ 947us) server_negotiation.cc:299] Negotiation successful
0807 10:51:22.371355 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":482,"threads_started":1}
I20260807 10:51:22.373351 19401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:22.357461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:33221 (local address 127.0.0.1:58576)
0807 10:51:22.358951 (+ 1490us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:22.358999 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:22.359274 (+ 275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:22.360525 (+ 1251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:22.360536 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:22.360580 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:51:22.361123 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:22.361141 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:22.363817 (+ 2676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:22.363826 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:22.367288 (+ 3462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:22.367315 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:22.370325 (+ 3010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:22.370331 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:22.370649 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:22.370756 (+ 107us) client_negotiation.cc:769] Sending connection context
0807 10:51:22.372240 (+ 1484us) client_negotiation.cc:240] Negotiation successful
0807 10:51:22.372586 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1241,"mutex_wait_us":51,"thread_start_us":1101,"threads_started":1}
I20260807 10:51:22.379627 19403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:22.358048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:37353 (local address 127.0.0.1:50620)
0807 10:51:22.369360 (+ 11312us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:22.369397 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:22.369560 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:22.370273 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:22.370277 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:22.370311 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:51:22.370869 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:22.370884 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:22.372545 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:22.372552 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:22.376651 (+ 4099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:22.376666 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:22.378670 (+ 2004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:22.378677 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:22.378868 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:22.378902 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 10:51:22.379022 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 10:51:22.379164 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11107,"mutex_wait_us":1,"thread_start_us":11602,"threads_started":1}
I20260807 10:51:22.379635 19405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:22.358612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50620 (local address 127.3.73.193:37353)
0807 10:51:22.359249 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:22.359257 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:22.369524 (+ 10267us) server_negotiation.cc:407] Connection header received
0807 10:51:22.369724 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:22.369736 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:22.370065 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:22.370290 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:51:22.371111 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:22.372313 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:22.376849 (+ 4536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:22.378439 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:22.378574 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:22.378815 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0807 10:51:22.379092 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 10:51:22.379269 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":530,"threads_started":1}
I20260807 10:51:22.379449 19085 tablet_service.cc:1467] Processing CreateTablet for tablet 37a44e5c3cb44c518d330c2ce9ebe3cf (DEFAULT_TABLE table=test-table [id=b903dc5300f448e2aafbc6b9520c8115]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:51:22.381227 19085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37a44e5c3cb44c518d330c2ce9ebe3cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:51:22.382591 19401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:22.359415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:38223 (local address 127.0.0.1:53428)
0807 10:51:22.373922 (+ 14507us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:22.373949 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:22.374086 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:22.375544 (+ 1458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:22.375549 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:22.375582 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:51:22.375778 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:22.375792 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:22.377605 (+ 1813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:22.377612 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:22.379536 (+ 1924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:22.379551 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:22.381777 (+ 2226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:22.381783 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:22.382008 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:22.382052 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 10:51:22.382152 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 10:51:22.382297 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14364,"thread_start_us":15426,"threads_started":1}
I20260807 10:51:22.383507 19407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:22.359817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53428 (local address 127.3.73.195:38223)
0807 10:51:22.374864 (+ 15047us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:22.374874 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:22.374908 (+ 34us) server_negotiation.cc:407] Connection header received
0807 10:51:22.375014 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:22.375030 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:22.375337 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:22.375562 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:51:22.375959 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:22.377408 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:22.380053 (+ 2645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:22.381272 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:22.381424 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:22.381664 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0807 10:51:22.382995 (+ 1331us) server_negotiation.cc:299] Negotiation successful
0807 10:51:22.383164 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14791,"thread_start_us":14568,"threads_started":1}
I20260807 10:51:22.384025 18874 tablet_service.cc:1467] Processing CreateTablet for tablet 37a44e5c3cb44c518d330c2ce9ebe3cf (DEFAULT_TABLE table=test-table [id=b903dc5300f448e2aafbc6b9520c8115]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:51:22.385434 18874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37a44e5c3cb44c518d330c2ce9ebe3cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:51:22.387339 19287 tablet_service.cc:1467] Processing CreateTablet for tablet 37a44e5c3cb44c518d330c2ce9ebe3cf (DEFAULT_TABLE table=test-table [id=b903dc5300f448e2aafbc6b9520c8115]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:51:22.388896 19287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37a44e5c3cb44c518d330c2ce9ebe3cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:51:22.397203 19408 tablet_bootstrap.cc:492] T 37a44e5c3cb44c518d330c2ce9ebe3cf P decdd2305ec143f683ca4743b8a63087: Bootstrap starting.
I20260807 10:51:22.400779 19408 tablet_bootstrap.cc:654] T 37a44e5c3cb44c518d330c2ce9ebe3cf P decdd2305ec143f683ca4743b8a63087: Neither blocks nor log segments found. Creating new log.
I20260807 10:51:22.401963 19408 log.cc:824] T 37a44e5c3cb44c518d330c2ce9ebe3cf P decdd2305ec143f683ca4743b8a63087: Log is configured to *not* fsync() on all Append() calls
I20260807 10:51:22.402962 19409 tablet_bootstrap.cc:492] T 37a44e5c3cb44c518d330c2ce9ebe3cf P ed15463bc70c42dfa8c753ac6b7dbbc9: Bootstrap starting.
I20260807 10:51:22.405421 19412 tablet_bootstrap.cc:492] T 37a44e5c3cb44c518d330c2ce9ebe3cf P eb2ca21f9547448d988820988f5355d8: Bootstrap starting.
I20260807 10:51:22.407006 19409 tablet_bootstrap.cc:654] T 37a44e5c3cb44c518d330c2ce9ebe3cf P ed15463bc70c42dfa8c753ac6b7dbbc9: Neither blocks nor log segments found. Creating new log.
I20260807 10:51:22.407148 19408 tablet_bootstrap.cc:492] T 37a44e5c3cb44c518d330c2ce9ebe3cf P decdd2305ec143f683ca4743b8a63087: No bootstrap required, opened a new log
I20260807 10:51:22.407461 19408 ts_tablet_manager.cc:1397] T 37a44e5c3cb44c518d330c2ce9ebe3cf P decdd2305ec143f683ca4743b8a63087: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.001s
I20260807 10:51:22.408324 19409 log.cc:824] T 37a44e5c3cb44c518d330c2ce9ebe3cf P ed15463bc70c42dfa8c753ac6b7dbbc9: Log is configured to *not* fsync() on all Append() calls
I20260807 10:51:22.408473 19412 tablet_bootstrap.cc:654] T 37a44e5c3cb44c518d330c2ce9ebe3cf P eb2ca21f9547448d988820988f5355d8: Neither blocks nor log segments found. Creating new log.
I20260807 10:51:22.409787 19412 log.cc:824] T 37a44e5c3cb44c518d330c2ce9ebe3cf P eb2ca21f9547448d988820988f5355d8: Log is configured to *not* fsync() on all Append() calls
I20260807 10:51:22.413115 19409 tablet_bootstrap.cc:492] T 37a44e5c3cb44c518d330c2ce9ebe3cf P ed15463bc70c42dfa8c753ac6b7dbbc9: No bootstrap required, opened a new log
I20260807 10:51:22.413410 19409 ts_tablet_manager.cc:1397] T 37a44e5c3cb44c518d330c2ce9ebe3cf P ed15463bc70c42dfa8c753ac6b7dbbc9: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260807 10:51:22.413574 19412 tablet_bootstrap.cc:492] T 37a44e5c3cb44c518d330c2ce9ebe3cf P eb2ca21f9547448d988820988f5355d8: No bootstrap required, opened a new log
I20260807 10:51:22.413842 19412 ts_tablet_manager.cc:1397] T 37a44e5c3cb44c518d330c2ce9ebe3cf P eb2ca21f9547448d988820988f5355d8: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 10:51:22.419328 19408 raft_consensus.cc:348] T 37a44e5c3cb44c518d330c2ce9ebe3cf P decdd2305ec143f683ca4743b8a63087 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed15463bc70c42dfa8c753ac6b7dbbc9" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 33221 } } peers { permanent_uuid: "eb2ca21f9547448d988820988f5355d8" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 38223 } } peers { permanent_uuid: "decdd2305ec143f683ca4743b8a63087" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 37353 } }
I20260807 10:51:22.419972 19408 raft_consensus.cc:374] T 37a44e5c3cb44c518d330c2ce9ebe3cf P decdd2305ec143f683ca4743b8a63087 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:51:22.420126 19408 raft_consensus.cc:729] T 37a44e5c3cb44c518d330c2ce9ebe3cf P decdd2305ec143f683ca4743b8a63087 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: decdd2305ec143f683ca4743b8a63087, State: Initialized, Role: FOLLOWER
I20260807 10:51:22.421015 19408 consensus_queue.cc:260] T 37a44e5c3cb44c518d330c2ce9ebe3cf P decdd2305ec143f683ca4743b8a63087 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed15463bc70c42dfa8c753ac6b7dbbc9" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 33221 } } peers { permanent_uuid: "eb2ca21f9547448d988820988f5355d8" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 38223 } } peers { permanent_uuid: "decdd2305ec143f683ca4743b8a63087" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 37353 } }
I20260807 10:51:22.421697 19409 raft_consensus.cc:348] T 37a44e5c3cb44c518d330c2ce9ebe3cf P ed15463bc70c42dfa8c753ac6b7dbbc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed15463bc70c42dfa8c753ac6b7dbbc9" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 33221 } } peers { permanent_uuid: "eb2ca21f9547448d988820988f5355d8" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 38223 } } peers { permanent_uuid: "decdd2305ec143f683ca4743b8a63087" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 37353 } }
I20260807 10:51:22.422307 19409 raft_consensus.cc:374] T 37a44e5c3cb44c518d330c2ce9ebe3cf P ed15463bc70c42dfa8c753ac6b7dbbc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:51:22.422464 19409 raft_consensus.cc:729] T 37a44e5c3cb44c518d330c2ce9ebe3cf P ed15463bc70c42dfa8c753ac6b7dbbc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed15463bc70c42dfa8c753ac6b7dbbc9, State: Initialized, Role: FOLLOWER
I20260807 10:51:22.423026 19408 ts_tablet_manager.cc:1428] T 37a44e5c3cb44c518d330c2ce9ebe3cf P decdd2305ec143f683ca4743b8a63087: Time spent starting tablet: real 0.015s user 0.010s sys 0.006s
I20260807 10:51:22.423264 19409 consensus_queue.cc:260] T 37a44e5c3cb44c518d330c2ce9ebe3cf P ed15463bc70c42dfa8c753ac6b7dbbc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed15463bc70c42dfa8c753ac6b7dbbc9" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 33221 } } peers { permanent_uuid: "eb2ca21f9547448d988820988f5355d8" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 38223 } } peers { permanent_uuid: "decdd2305ec143f683ca4743b8a63087" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 37353 } }
I20260807 10:51:22.424391 19180 heartbeater.cc:503] Master 127.3.73.254:42229 was elected leader, sending a full tablet report...
I20260807 10:51:22.425033 19409 ts_tablet_manager.cc:1428] T 37a44e5c3cb44c518d330c2ce9ebe3cf P ed15463bc70c42dfa8c753ac6b7dbbc9: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260807 10:51:22.425119 19412 raft_consensus.cc:348] T 37a44e5c3cb44c518d330c2ce9ebe3cf P eb2ca21f9547448d988820988f5355d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed15463bc70c42dfa8c753ac6b7dbbc9" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 33221 } } peers { permanent_uuid: "eb2ca21f9547448d988820988f5355d8" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 38223 } } peers { permanent_uuid: "decdd2305ec143f683ca4743b8a63087" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 37353 } }
I20260807 10:51:22.425676 19412 raft_consensus.cc:374] T 37a44e5c3cb44c518d330c2ce9ebe3cf P eb2ca21f9547448d988820988f5355d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:51:22.425803 19412 raft_consensus.cc:729] T 37a44e5c3cb44c518d330c2ce9ebe3cf P eb2ca21f9547448d988820988f5355d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb2ca21f9547448d988820988f5355d8, State: Initialized, Role: FOLLOWER
I20260807 10:51:22.426469 19412 consensus_queue.cc:260] T 37a44e5c3cb44c518d330c2ce9ebe3cf P eb2ca21f9547448d988820988f5355d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed15463bc70c42dfa8c753ac6b7dbbc9" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 33221 } } peers { permanent_uuid: "eb2ca21f9547448d988820988f5355d8" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 38223 } } peers { permanent_uuid: "decdd2305ec143f683ca4743b8a63087" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 37353 } }
I20260807 10:51:22.427778 19385 heartbeater.cc:503] Master 127.3.73.254:42229 was elected leader, sending a full tablet report...
I20260807 10:51:22.428283 19412 ts_tablet_manager.cc:1428] T 37a44e5c3cb44c518d330c2ce9ebe3cf P eb2ca21f9547448d988820988f5355d8: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
W20260807 10:51:22.449819 19181 tablet.cc:2369] T 37a44e5c3cb44c518d330c2ce9ebe3cf P ed15463bc70c42dfa8c753ac6b7dbbc9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:51:22.472185 19418 raft_consensus.cc:482] T 37a44e5c3cb44c518d330c2ce9ebe3cf P ed15463bc70c42dfa8c753ac6b7dbbc9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 10:51:22.472559 19418 raft_consensus.cc:504] T 37a44e5c3cb44c518d330c2ce9ebe3cf P ed15463bc70c42dfa8c753ac6b7dbbc9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed15463bc70c42dfa8c753ac6b7dbbc9" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 33221 } } peers { permanent_uuid: "eb2ca21f9547448d988820988f5355d8" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 38223 } } peers { permanent_uuid: "decdd2305ec143f683ca4743b8a63087" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 37353 } }
I20260807 10:51:22.474408 19418 leader_election.cc:288] T 37a44e5c3cb44c518d330c2ce9ebe3cf P ed15463bc70c42dfa8c753ac6b7dbbc9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb2ca21f9547448d988820988f5355d8 (127.3.73.195:38223), decdd2305ec143f683ca4743b8a63087 (127.3.73.193:37353)
I20260807 10:51:22.482676 19407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:22.474732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:52953 (local address 127.3.73.195:38223)
0807 10:51:22.474964 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:22.474973 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:22.476074 (+ 1101us) server_negotiation.cc:407] Connection header received
0807 10:51:22.476303 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:22.476308 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:22.476377 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:22.476475 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:51:22.477043 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:22.477793 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:22.480328 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:22.481476 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:22.481741 (+ 265us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:22.481825 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 10:51:22.482156 (+ 331us) server_negotiation.cc:299] Negotiation successful
0807 10:51:22.482285 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 10:51:22.482861 19422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:22.474629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:38223 (local address 127.3.73.194:52953)
0807 10:51:22.475950 (+ 1321us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:22.475967 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:22.476149 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:22.476492 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:22.476496 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:22.476522 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:51:22.476897 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:22.476907 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:22.477927 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:22.477938 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:22.480148 (+ 2210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:22.480160 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:22.481788 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:22.481795 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:22.481978 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:22.482003 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 10:51:22.482116 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 10:51:22.482259 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1175,"thread_start_us":966,"threads_started":1}
I20260807 10:51:22.484268 19319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37a44e5c3cb44c518d330c2ce9ebe3cf" candidate_uuid: "ed15463bc70c42dfa8c753ac6b7dbbc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb2ca21f9547448d988820988f5355d8" is_pre_election: true
I20260807 10:51:22.485370 19319 raft_consensus.cc:2454] T 37a44e5c3cb44c518d330c2ce9ebe3cf P eb2ca21f9547448d988820988f5355d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed15463bc70c42dfa8c753ac6b7dbbc9 in term 0.
I20260807 10:51:22.486605 19013 leader_election.cc:302] T 37a44e5c3cb44c518d330c2ce9ebe3cf P ed15463bc70c42dfa8c753ac6b7dbbc9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eb2ca21f9547448d988820988f5355d8, ed15463bc70c42dfa8c753ac6b7dbbc9; no voters:
I20260807 10:51:22.487112 19418 raft_consensus.cc:2781] T 37a44e5c3cb44c518d330c2ce9ebe3cf P ed15463bc70c42dfa8c753ac6b7dbbc9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 10:51:22.487170 19405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:22.475151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:49181 (local address 127.3.73.193:37353)
0807 10:51:22.475440 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:22.475448 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:22.476232 (+ 784us) server_negotiation.cc:407] Connection header received
0807 10:51:22.476613 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:22.476634 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:22.476756 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:22.476846 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:51:22.478114 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:22.479280 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:22.483653 (+ 4373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:22.485782 (+ 2129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:22.486010 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:22.486099 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 10:51:22.486699 (+ 600us) server_negotiation.cc:299] Negotiation successful
0807 10:51:22.486835 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 10:51:22.487349 19418 raft_consensus.cc:482] T 37a44e5c3cb44c518d330c2ce9ebe3cf P ed15463bc70c42dfa8c753ac6b7dbbc9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 10:51:22.487636 19421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:22.474725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:37353 (local address 127.3.73.194:49181)
0807 10:51:22.476123 (+ 1398us) negotiation.cc:107] Waiting for socket to connect
0807 10:51:22.476150 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 10:51:22.476394 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:51:22.477111 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:51:22.477119 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:51:22.477159 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:51:22.477815 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:22.477834 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:22.479517 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:22.479534 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:22.483384 (+ 3850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:51:22.483400 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:51:22.486373 (+ 2973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:51:22.486378 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:51:22.486521 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:22.486545 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 10:51:22.486659 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 10:51:22.486813 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1236,"thread_start_us":683,"threads_started":1}
I20260807 10:51:22.487654 19418 raft_consensus.cc:3037] T 37a44e5c3cb44c518d330c2ce9ebe3cf P ed15463bc70c42dfa8c753ac6b7dbbc9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:51:22.490136 18908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37a44e5c3cb44c518d330c2ce9ebe3cf" candidate_uuid: "ed15463bc70c42dfa8c753ac6b7dbbc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "decdd2305ec143f683ca4743b8a63087" is_pre_election: true
I20260807 10:51:22.490520 19418 raft_consensus.cc:504] T 37a44e5c3cb44c518d330c2ce9ebe3cf P ed15463bc70c42dfa8c753ac6b7dbbc9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed15463bc70c42dfa8c753ac6b7dbbc9" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 33221 } } peers { permanent_uuid: "eb2ca21f9547448d988820988f5355d8" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 38223 } } peers { permanent_uuid: "decdd2305ec143f683ca4743b8a63087" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 37353 } }
I20260807 10:51:22.491088 18908 raft_consensus.cc:2454] T 37a44e5c3cb44c518d330c2ce9ebe3cf P decdd2305ec143f683ca4743b8a63087 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ed15463bc70c42dfa8c753ac6b7dbbc9 in term 0.
I20260807 10:51:22.491364 19418 leader_election.cc:288] T 37a44e5c3cb44c518d330c2ce9ebe3cf P ed15463bc70c42dfa8c753ac6b7dbbc9 [CANDIDATE]: Term 1 election: Requested vote from peers eb2ca21f9547448d988820988f5355d8 (127.3.73.195:38223), decdd2305ec143f683ca4743b8a63087 (127.3.73.193:37353)
I20260807 10:51:22.491829 19319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37a44e5c3cb44c518d330c2ce9ebe3cf" candidate_uuid: "ed15463bc70c42dfa8c753ac6b7dbbc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb2ca21f9547448d988820988f5355d8"
I20260807 10:51:22.492098 19319 raft_consensus.cc:3037] T 37a44e5c3cb44c518d330c2ce9ebe3cf P eb2ca21f9547448d988820988f5355d8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:51:22.492139 18908 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37a44e5c3cb44c518d330c2ce9ebe3cf" candidate_uuid: "ed15463bc70c42dfa8c753ac6b7dbbc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "decdd2305ec143f683ca4743b8a63087"
I20260807 10:51:22.492430 18908 raft_consensus.cc:3037] T 37a44e5c3cb44c518d330c2ce9ebe3cf P decdd2305ec143f683ca4743b8a63087 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:51:22.495508 19319 raft_consensus.cc:2454] T 37a44e5c3cb44c518d330c2ce9ebe3cf P eb2ca21f9547448d988820988f5355d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed15463bc70c42dfa8c753ac6b7dbbc9 in term 1.
I20260807 10:51:22.496054 19013 leader_election.cc:302] T 37a44e5c3cb44c518d330c2ce9ebe3cf P ed15463bc70c42dfa8c753ac6b7dbbc9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: eb2ca21f9547448d988820988f5355d8, ed15463bc70c42dfa8c753ac6b7dbbc9; no voters:
I20260807 10:51:22.496604 19418 raft_consensus.cc:2781] T 37a44e5c3cb44c518d330c2ce9ebe3cf P ed15463bc70c42dfa8c753ac6b7dbbc9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:51:22.497254 18908 raft_consensus.cc:2454] T 37a44e5c3cb44c518d330c2ce9ebe3cf P decdd2305ec143f683ca4743b8a63087 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ed15463bc70c42dfa8c753ac6b7dbbc9 in term 1.
I20260807 10:51:22.497534 19418 raft_consensus.cc:686] T 37a44e5c3cb44c518d330c2ce9ebe3cf P ed15463bc70c42dfa8c753ac6b7dbbc9 [term 1 LEADER]: Becoming Leader. State: Replica: ed15463bc70c42dfa8c753ac6b7dbbc9, State: Running, Role: LEADER
I20260807 10:51:22.498235 19418 consensus_queue.cc:237] T 37a44e5c3cb44c518d330c2ce9ebe3cf P ed15463bc70c42dfa8c753ac6b7dbbc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed15463bc70c42dfa8c753ac6b7dbbc9" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 33221 } } peers { permanent_uuid: "eb2ca21f9547448d988820988f5355d8" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 38223 } } peers { permanent_uuid: "decdd2305ec143f683ca4743b8a63087" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 37353 } }
I20260807 10:51:22.505009 18674 catalog_manager.cc:5495] T 37a44e5c3cb44c518d330c2ce9ebe3cf P ed15463bc70c42dfa8c753ac6b7dbbc9 reported cstate change: term changed from 0 to 1, leader changed from <none> to ed15463bc70c42dfa8c753ac6b7dbbc9 (127.3.73.194). New cstate: current_term: 1 leader_uuid: "ed15463bc70c42dfa8c753ac6b7dbbc9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed15463bc70c42dfa8c753ac6b7dbbc9" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 33221 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb2ca21f9547448d988820988f5355d8" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 38223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "decdd2305ec143f683ca4743b8a63087" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 37353 } health_report { overall_health: UNKNOWN } } }
W20260807 10:51:22.533054 19386 tablet.cc:2369] T 37a44e5c3cb44c518d330c2ce9ebe3cf P eb2ca21f9547448d988820988f5355d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:51:22.540323 19402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:22.533560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58588 (local address 127.3.73.194:33221)
0807 10:51:22.533909 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:22.533916 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:22.534060 (+ 144us) server_negotiation.cc:407] Connection header received
0807 10:51:22.534305 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:22.534309 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:22.534393 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:22.534560 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:51:22.534962 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:22.536100 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:22.537678 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:22.538296 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:22.538411 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:22.538971 (+ 560us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:51:22.539517 (+ 546us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:51:22.539633 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 10:51:22.539784 (+ 151us) server_negotiation.cc:299] Negotiation successful
0807 10:51:22.539939 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 10:51:22.551771 19319 raft_consensus.cc:1261] T 37a44e5c3cb44c518d330c2ce9ebe3cf P eb2ca21f9547448d988820988f5355d8 [term 1 FOLLOWER]: Refusing update from remote peer ed15463bc70c42dfa8c753ac6b7dbbc9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:51:22.551932 18908 raft_consensus.cc:1261] T 37a44e5c3cb44c518d330c2ce9ebe3cf P decdd2305ec143f683ca4743b8a63087 [term 1 FOLLOWER]: Refusing update from remote peer ed15463bc70c42dfa8c753ac6b7dbbc9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:51:22.553200 19423 consensus_queue.cc:1035] T 37a44e5c3cb44c518d330c2ce9ebe3cf P ed15463bc70c42dfa8c753ac6b7dbbc9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb2ca21f9547448d988820988f5355d8" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 38223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:51:22.553886 19418 consensus_queue.cc:1035] T 37a44e5c3cb44c518d330c2ce9ebe3cf P ed15463bc70c42dfa8c753ac6b7dbbc9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "decdd2305ec143f683ca4743b8a63087" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 37353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:51:22.570994 19426 mvcc.cc:205] Tried to move back new op lower bound from 7315865118917558272 to 7315865118717337600. Current Snapshot: MvccSnapshot[applied={T|T < 7315865118917558272}]
I20260807 10:51:22.572501 19433 mvcc.cc:205] Tried to move back new op lower bound from 7315865118917558272 to 7315865118717337600. Current Snapshot: MvccSnapshot[applied={T|T < 7315865118917558272}]
I20260807 10:51:22.587100 18674 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:51:22.587417 18674 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47180:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:51:22.591430 18674 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 597634295a16444ab7c7f5acc99bd3db: Sending DeleteTablet for 3 replicas of tablet 37a44e5c3cb44c518d330c2ce9ebe3cf
I20260807 10:51:22.592908 19085 tablet_service.cc:1514] Processing DeleteTablet for tablet 37a44e5c3cb44c518d330c2ce9ebe3cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:51:22 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:58576
I20260807 10:51:22.593076 19287 tablet_service.cc:1514] Processing DeleteTablet for tablet 37a44e5c3cb44c518d330c2ce9ebe3cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:51:22 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:53428
I20260807 10:51:22.593282 18874 tablet_service.cc:1514] Processing DeleteTablet for tablet 37a44e5c3cb44c518d330c2ce9ebe3cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:51:22 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:50620
I20260807 10:51:22.595188 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 18779
I20260807 10:51:22.627322 19450 tablet_replica.cc:330] T 37a44e5c3cb44c518d330c2ce9ebe3cf P decdd2305ec143f683ca4743b8a63087: stopping tablet replica
I20260807 10:51:22.627877 19450 raft_consensus.cc:2229] T 37a44e5c3cb44c518d330c2ce9ebe3cf P decdd2305ec143f683ca4743b8a63087 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:51:22.628587 19448 tablet_replica.cc:330] T 37a44e5c3cb44c518d330c2ce9ebe3cf P ed15463bc70c42dfa8c753ac6b7dbbc9: stopping tablet replica
I20260807 10:51:22.628585 19450 raft_consensus.cc:2258] T 37a44e5c3cb44c518d330c2ce9ebe3cf P decdd2305ec143f683ca4743b8a63087 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:51:22.629333 19449 tablet_replica.cc:330] T 37a44e5c3cb44c518d330c2ce9ebe3cf P eb2ca21f9547448d988820988f5355d8: stopping tablet replica
I20260807 10:51:22.629753 19449 raft_consensus.cc:2229] T 37a44e5c3cb44c518d330c2ce9ebe3cf P eb2ca21f9547448d988820988f5355d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:51:22.630033 19448 raft_consensus.cc:2229] T 37a44e5c3cb44c518d330c2ce9ebe3cf P ed15463bc70c42dfa8c753ac6b7dbbc9 [term 1 LEADER]: Raft consensus shutting down.
I20260807 10:51:22.630326 19449 raft_consensus.cc:2258] T 37a44e5c3cb44c518d330c2ce9ebe3cf P eb2ca21f9547448d988820988f5355d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:51:22.632540 19449 ts_tablet_manager.cc:1905] T 37a44e5c3cb44c518d330c2ce9ebe3cf P eb2ca21f9547448d988820988f5355d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:51:22.632614 19450 ts_tablet_manager.cc:1905] T 37a44e5c3cb44c518d330c2ce9ebe3cf P decdd2305ec143f683ca4743b8a63087: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 10:51:22 dist-test-slave-x5fp krb5kdc[18609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.193@KRBTEST.COM
I20260807 10:51:22.634739 19448 raft_consensus.cc:2258] T 37a44e5c3cb44c518d330c2ce9ebe3cf P ed15463bc70c42dfa8c753ac6b7dbbc9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:51:22.640501 19449 ts_tablet_manager.cc:1918] T 37a44e5c3cb44c518d330c2ce9ebe3cf P eb2ca21f9547448d988820988f5355d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:51:22.640506 19450 ts_tablet_manager.cc:1918] T 37a44e5c3cb44c518d330c2ce9ebe3cf P decdd2305ec143f683ca4743b8a63087: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:51:22.640681 19450 log.cc:1196] T 37a44e5c3cb44c518d330c2ce9ebe3cf P decdd2305ec143f683ca4743b8a63087: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-0/wal/wals/37a44e5c3cb44c518d330c2ce9ebe3cf
I20260807 10:51:22.641232 19449 log.cc:1196] T 37a44e5c3cb44c518d330c2ce9ebe3cf P eb2ca21f9547448d988820988f5355d8: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-2/wal/wals/37a44e5c3cb44c518d330c2ce9ebe3cf
I20260807 10:51:22.641319 19450 ts_tablet_manager.cc:1939] T 37a44e5c3cb44c518d330c2ce9ebe3cf P decdd2305ec143f683ca4743b8a63087: Deleting consensus metadata
I20260807 10:51:22.641948 19449 ts_tablet_manager.cc:1939] T 37a44e5c3cb44c518d330c2ce9ebe3cf P eb2ca21f9547448d988820988f5355d8: Deleting consensus metadata
I20260807 10:51:22.643549 18652 catalog_manager.cc:4841] TS decdd2305ec143f683ca4743b8a63087 (127.3.73.193:37353): tablet 37a44e5c3cb44c518d330c2ce9ebe3cf (table test-table [id=b903dc5300f448e2aafbc6b9520c8115]) successfully deleted
I20260807 10:51:22.644120 18650 catalog_manager.cc:4841] TS eb2ca21f9547448d988820988f5355d8 (127.3.73.195:38223): tablet 37a44e5c3cb44c518d330c2ce9ebe3cf (table test-table [id=b903dc5300f448e2aafbc6b9520c8115]) successfully deleted
I20260807 10:51:22.645277 19448 ts_tablet_manager.cc:1905] T 37a44e5c3cb44c518d330c2ce9ebe3cf P ed15463bc70c42dfa8c753ac6b7dbbc9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:51:22.647948 19405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:22.597025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50624 (local address 127.3.73.193:37353)
0807 10:51:22.597219 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:22.597227 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:22.618942 (+ 21715us) server_negotiation.cc:407] Connection header received
0807 10:51:22.619522 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:22.619528 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:22.619623 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:22.619714 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:22.622291 (+ 2577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:22.623851 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:22.628075 (+ 4224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:22.628917 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:22.629061 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:22.636477 (+ 7416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:51:22.636492 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:51:22.636510 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:51:22.636558 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:51:22.644216 (+ 7658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:51:22.645320 (+ 1104us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:51:22.645330 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:51:22.645334 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:51:22.645525 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:51:22.646050 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:51:22.646056 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:51:22.646060 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:51:22.646577 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:51:22.646926 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0807 10:51:22.647282 (+ 356us) server_negotiation.cc:299] Negotiation successful
0807 10:51:22.647548 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 10:51:22.648669 18967 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:51:22.653764 19448 ts_tablet_manager.cc:1918] T 37a44e5c3cb44c518d330c2ce9ebe3cf P ed15463bc70c42dfa8c753ac6b7dbbc9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:51:22.653981 19448 log.cc:1196] T 37a44e5c3cb44c518d330c2ce9ebe3cf P ed15463bc70c42dfa8c753ac6b7dbbc9: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786099767749326-3367-0/minicluster-data/ts-1/wal/wals/37a44e5c3cb44c518d330c2ce9ebe3cf
I20260807 10:51:22.654475 19448 ts_tablet_manager.cc:1939] T 37a44e5c3cb44c518d330c2ce9ebe3cf P ed15463bc70c42dfa8c753ac6b7dbbc9: Deleting consensus metadata
I20260807 10:51:22.655969 18650 catalog_manager.cc:4841] TS ed15463bc70c42dfa8c753ac6b7dbbc9 (127.3.73.194:33221): tablet 37a44e5c3cb44c518d330c2ce9ebe3cf (table test-table [id=b903dc5300f448e2aafbc6b9520c8115]) successfully deleted
I20260807 10:51:22.794107 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 18779
I20260807 10:51:22.824925 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 18982
Aug 07 10:51:22 dist-test-slave-x5fp krb5kdc[18609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.194@KRBTEST.COM
I20260807 10:51:22.843277 19402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:22.825770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58590 (local address 127.3.73.194:33221)
0807 10:51:22.825980 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:22.825984 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:22.826056 (+ 72us) server_negotiation.cc:407] Connection header received
0807 10:51:22.826237 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:22.826241 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:22.826317 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:22.826407 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:22.828168 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:22.829097 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:22.830942 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:22.831400 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:22.831498 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:22.836669 (+ 5171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:51:22.836685 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:51:22.836702 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:51:22.836808 (+ 106us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:51:22.840200 (+ 3392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:51:22.841003 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:51:22.841011 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:51:22.841013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:51:22.841175 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:51:22.841589 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:51:22.841595 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:51:22.841598 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:51:22.842003 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:51:22.842358 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0807 10:51:22.842655 (+ 297us) server_negotiation.cc:299] Negotiation successful
0807 10:51:22.842887 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 10:51:22.843957 19172 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:51:22.963734 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 18982
I20260807 10:51:22.994719 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 19190
Aug 07 10:51:22 dist-test-slave-x5fp krb5kdc[18609](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.195@KRBTEST.COM
I20260807 10:51:23.011896 19455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:51:22.995552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53432 (local address 127.3.73.195:38223)
0807 10:51:22.996054 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0807 10:51:22.996059 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:51:22.996075 (+ 16us) server_negotiation.cc:407] Connection header received
0807 10:51:22.996132 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:51:22.996135 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:51:22.996194 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:51:22.996330 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:51:22.997930 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:22.998689 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:23.000043 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:51:23.000437 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:51:23.000581 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:51:23.005479 (+ 4898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:51:23.005495 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:51:23.005511 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:51:23.005557 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:51:23.008855 (+ 3298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:51:23.009822 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:51:23.009832 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:51:23.009835 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:51:23.010008 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:51:23.010429 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:51:23.010440 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:51:23.010442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:51:23.010887 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:51:23.011130 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0807 10:51:23.011407 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 10:51:23.011589 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":258,"threads_started":1}
I20260807 10:51:23.012795 19378 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:51:23.105142 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 19190
I20260807 10:51:23.130714 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 18627
I20260807 10:51:23.131925 18729 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:51:23.248159 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 18627
2026-08-07T10:51:23Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4975 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:51:23 dist-test-slave-x5fp krb5kdc[19461](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:51:23 dist-test-slave-x5fp krb5kdc[19461](info): set up 2 sockets
Aug 07 10:51:23 dist-test-slave-x5fp krb5kdc[19461](info): commencing operation
krb5kdc: starting...
W20260807 10:51:24.588256 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.286s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:51:24 dist-test-slave-x5fp krb5kdc[19461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:51:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:51:24Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.3.73.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.3.73.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.3.73.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/rangeradmin_127.3.73.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.3.73.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/rangeradmin_127.3.73.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.3.73.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.3.73.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.3.73.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/rangerlookup_127.3.73.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.3.73.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/rangerlookup_127.3.73.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.3.73.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.3.73.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.3.73.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/HTTP_127.3.73.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.3.73.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/HTTP_127.3.73.212@KRBTEST.COM.keytab.
I20260807 10:51:25.057631 3367 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskRBZOut/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/postgres -l logfile start
2026-08-07 10:51:28.095 UTC [19507] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 10:51:28.095 UTC [19507] LOG: listening on IPv4 address "127.3.73.212", port 59413
2026-08-07 10:51:28.099 UTC [19507] LOG: listening on Unix socket "/tmp/.s.PGSQL.59413"
2026-08-07 10:51:28.127 UTC [19510] LOG: database system was shut down at 2026-08-07 10:51:26 UTC
2026-08-07 10:51:28.131 UTC [19507] LOG: database system is ready to accept connections
I20260807 10:51:29.386372 3367 mini_postgres.cc:96] Postgres bound to 59413
2026-08-07 10:51:29.420 UTC [19524] FATAL: database "slave" does not exist
127.3.73.212:59413 - accepting connections
I20260807 10:51:29.421444 3367 mini_ranger.cc:161] Starting Ranger...
I20260807 10:51:29.475697 3367 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 10:51:30.065596 3367 mini_ranger.cc:87] Created ranger Postgres database
I20260807 10:51:30.065770 3367 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ranger-admin
2026-08-07 10:51:30,522 [I] DB FLAVOR :POSTGRES
2026-08-07 10:51:30,522 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 10:51:30,522 [I] Checking connection..
2026-08-07 10:51:30,523 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:59413/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 10:51:30,817 [I] Checking connection passed.
2026-08-07 10:51:30,817 [I] --------- Verifying version history table ---------
2026-08-07 10:51:30,817 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:59413/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 10:51:31,090 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 10:51:31,091 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:59413/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 10:51:31,372 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 10:51:31,372 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 10:51:31,372 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:59413/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 10:51:31.619 UTC [19632] WARNING: there is no transaction in progress
2026-08-07 10:51:31,628 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 10:51:31,629 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:59413/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 10:51:31,894 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 10:51:31,895 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 10:51:31,895 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:59413/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 10:51:32,157 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:59413/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 10:51:32,458 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:59413/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-x5fp.c.gcp-upstream.internal','N') ;"
2026-08-07 10:51:32,708 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:59413/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 10:51:32,965 [I] Table x_portal_user does not exist in database ranger
2026-08-07 10:51:32,965 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:59413/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 10:51:33,227 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 10:51:33,228 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:59413/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 10:51:33,503 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:59413/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 10:51:33,777 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 10:51:33,778 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:59413/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 10:51:34.892 UTC [19805] WARNING: there is no transaction in progress
2026-08-07 10:51:34.912 UTC [19805] WARNING: there is no transaction in progress
2026-08-07 10:51:34.931 UTC [19805] WARNING: there is no transaction in progress
2026-08-07 10:51:34.949 UTC [19805] WARNING: there is no transaction in progress
2026-08-07 10:51:34.970 UTC [19805] WARNING: there is no transaction in progress
2026-08-07 10:51:34.993 UTC [19805] WARNING: there is no transaction in progress
2026-08-07 10:51:35.070 UTC [19805] WARNING: there is no transaction in progress
2026-08-07 10:51:35.078 UTC [19805] WARNING: there is no transaction in progress
2026-08-07 10:51:35.094 UTC [19805] WARNING: there is no transaction in progress
2026-08-07 10:51:35.106 UTC [19805] WARNING: there is no transaction in progress
2026-08-07 10:51:35.118 UTC [19805] WARNING: there is no transaction in progress
2026-08-07 10:51:35.133 UTC [19805] WARNING: there is no transaction in progress
2026-08-07 10:51:35.144 UTC [19805] WARNING: there is no transaction in progress
2026-08-07 10:51:35.152 UTC [19805] WARNING: there is no transaction in progress
2026-08-07 10:51:35.161 UTC [19805] WARNING: there is no transaction in progress
2026-08-07 10:51:35.170 UTC [19805] WARNING: there is no transaction in progress
2026-08-07 10:51:35,895 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 10:51:35,896 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:59413/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 10:51:36,164 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 10:51:36,165 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:59413/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 10:51:36,446 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 10:51:36,447 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:59413/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 10:51:36,717 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 10:51:36,718 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:59413/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 10:51:36,995 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:59413/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 10:51:37,270 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:59413/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-x5fp.c.gcp-upstream.internal';"
2026-08-07 10:51:37,538 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 10:51:37,539 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:59413/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 10:51:37,812 [I] DB_PATCHES have already been applied
I20260807 10:51:37.822292 3367 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ranger-admin:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskRBZOut/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 10:51:37.822454 3367 mini_ranger.cc:193] Using host: 127.3.73.212
I20260807 10:51:37.858469 3367 mini_ranger.cc:227] Ranger admin URL: http://127.3.73.212:39899
Aug 07, 2026 10:51:38 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 10:51:38 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 10:51:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 10:51:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 10:51:38 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 10:51:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 10:51:39 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 10:51:39.303488 3367 mini_ranger.cc:160] Time spent starting Ranger: real 9.882s user 0.000s sys 0.091s
Aug 07, 2026 10:52:03 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 10:52:03 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Aug 07, 2026 10:52:03 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 10:52:03 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 10:52:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 10:52:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 10:52:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 10:52:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 10:52:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 10:52:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 10:52:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 10:52:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 10:52:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 10:52:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 10:52:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 10:52:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 10:52:03 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 10:52:04 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 10:52:04.995409 3367 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.3.73.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.3.73.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.3.73.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/rangerkms_127.3.73.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.3.73.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/rangerkms_127.3.73.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.3.73.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.3.73.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.3.73.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/HTTP_127.3.73.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.3.73.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/HTTP_127.3.73.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 10:52:05 dist-test-slave-x5fp krb5kdc[19461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099925, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 10:52:05.275279 3367 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 10:52:05.311914 3367 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 10:52:06.527168 3367 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 10:52:06.527370 3367 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ranger-kms
I20260807 10:52:06.527504 3367 mini_ranger_kms.cc:227] Using postgres at 127.3.73.212:59413
2026-08-07 10:52:07,141 [I] DB FLAVOR :POSTGRES
2026-08-07 10:52:07,141 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 10:52:07,141 [I] Checking connection
2026-08-07 10:52:07,142 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:59413/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 10:52:07,415 [I] connection success
2026-08-07 10:52:07,415 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 10:52:07,415 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 10:52:07,415 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:59413/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 10:52:07,700 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 10:52:07,701 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 10:52:07,701 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 10:52:07,701 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:59413/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 10:52:07,999 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 10:52:08.563822 3367 mini_ranger_kms.cc:310] Created kms service
I20260807 10:52:08.798220 3367 mini_ranger_kms.cc:326] Created kudu user
I20260807 10:52:08.854475 3367 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 10:52:08 dist-test-slave-x5fp krb5kdc[19461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099925, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.3.73.212@KRBTEST.COM
I20260807 10:52:09.341842 3367 mini_ranger_kms.cc:384] Added ranger policy
I20260807 10:52:09.343568 3367 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ranger-kms:/tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 10:52:09.343647 3367 mini_ranger_kms.cc:240] Using host: 127.3.73.212
I20260807 10:52:09.375198 3367 mini_ranger_kms.cc:276] Ranger KMS PID: 20080
I20260807 10:52:09.375440 3367 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.3.73.212:59437
Aug 07, 2026 10:52:10 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 10:52:10 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 10:52:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 10:52:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 10:52:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 10:52:11 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 10:52:11 dist-test-slave-x5fp krb5kdc[19461](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786099931, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.73.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:52:12.261139 3367 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.986s user 0.158s sys 0.513s
I20260807 10:52:12.261529 3367 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 10:52:12.261785 3367 mini_ranger_kms.cc:401] 127.3.73.212:59437/kms/v1/keys
Aug 07 10:52:12 dist-test-slave-x5fp krb5kdc[19461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099931, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.73.212@KRBTEST.COM for HTTP/127.3.73.212@KRBTEST.COM
Aug 07 10:52:12 dist-test-slave-x5fp krb5kdc[19461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099931, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.73.212@KRBTEST.COM for HTTP/127.3.73.212@KRBTEST.COM
Aug 07 10:52:15 dist-test-slave-x5fp krb5kdc[19461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.3.73.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
I20260807 10:52:15.546444 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:35913
--webserver_interface=127.3.73.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:44891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:35913
--ranger_config_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.73.212:59437/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:52:16.126425 20166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:52:16.126881 20166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:52:16.127162 20166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:52:16.137661 20166 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:52:16.137925 20166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:52:16.138052 20166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:52:16.138211 20166 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:52:16.138345 20166 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:52:16.152292 20166 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:44891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/master-0/wal
--ranger_kms_url=127.3.73.212:59437/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:35913
--ranger_config_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:35913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:52:16.154110 20166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:52:16.155766 20166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:52:16.171177 20172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:52:16.171293 20177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:52:16.171186 20173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:52:16.172502 20166 server_base.cc:1034] running on GCE node
I20260807 10:52:16.173496 20166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:52:16.175108 20166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:52:16.176416 20166 hybrid_clock.cc:648] HybridClock initialized: now 1786099936176372 us; error 50 us; skew 500 ppm
I20260807 10:52:16.176949 20166 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:52:16 dist-test-slave-x5fp krb5kdc[19461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099936, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:52:16.180780 20166 init.cc:377] Logged in from keytab as kudu/127.3.73.254@KRBTEST.COM (short username kudu)
I20260807 10:52:16.182893 20166 webserver.cc:460] Webserver started at http://127.3.73.254:34249/ using document root <none> and password file <none>
I20260807 10:52:16.183768 20166 fs_manager.cc:359] Metadata directory not provided
I20260807 10:52:16.183976 20166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:52:16.184405 20166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 10:52:16 dist-test-slave-x5fp krb5kdc[19461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099936, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for HTTP/127.3.73.212@KRBTEST.COM
I20260807 10:52:16.267676 20166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "7843cf9c69554d34872bb8cfe65fd06a"
format_stamp: "Formatted at 2026-08-07 10:52:16 on dist-test-slave-x5fp"
server_key: "8a67c69502f85ab75c5543d38ade3232"
server_key_iv: "cdc3db1835a3af4353bb582859b23031"
server_key_version: "kuduclusterkey@0"
I20260807 10:52:16.268819 20166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "7843cf9c69554d34872bb8cfe65fd06a"
format_stamp: "Formatted at 2026-08-07 10:52:16 on dist-test-slave-x5fp"
server_key: "8a67c69502f85ab75c5543d38ade3232"
server_key_iv: "cdc3db1835a3af4353bb582859b23031"
server_key_version: "kuduclusterkey@0"
I20260807 10:52:16.276607 20166 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260807 10:52:16.301065 20188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:52:16.303148 20166 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.001s
I20260807 10:52:16.303436 20166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "7843cf9c69554d34872bb8cfe65fd06a"
format_stamp: "Formatted at 2026-08-07 10:52:16 on dist-test-slave-x5fp"
server_key: "8a67c69502f85ab75c5543d38ade3232"
server_key_iv: "cdc3db1835a3af4353bb582859b23031"
server_key_version: "kuduclusterkey@0"
I20260807 10:52:16.303913 20166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:52:16.343546 20166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:52:16.349129 20166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:52:16.349694 20166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:52:16.392531 20166 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:35913
I20260807 10:52:16.392561 20254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:35913 every 8 connection(s)
I20260807 10:52:16.394625 20166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:52:16.396615 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 20166
I20260807 10:52:16.396929 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:52:16.397948 3367 external_mini_cluster.cc:1428] Setting key a04decbf28d2709d767f69f9a0f41818
I20260807 10:52:16.401217 20255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 10:52:16 dist-test-slave-x5fp krb5kdc[19461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:52:16.417979 20255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7843cf9c69554d34872bb8cfe65fd06a: Bootstrap starting.
I20260807 10:52:16.423098 20255 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7843cf9c69554d34872bb8cfe65fd06a: Neither blocks nor log segments found. Creating new log.
I20260807 10:52:16.425159 20255 log.cc:824] T 00000000000000000000000000000000 P 7843cf9c69554d34872bb8cfe65fd06a: Log is configured to *not* fsync() on all Append() calls
I20260807 10:52:16.428128 20260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:52:16.400499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36828 (local address 127.3.73.254:35913)
0807 10:52:16.402354 (+ 1855us) server_negotiation.cc:206] Beginning negotiation
0807 10:52:16.402372 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 10:52:16.402435 (+ 63us) server_negotiation.cc:407] Connection header received
0807 10:52:16.404973 (+ 2538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:52:16.405000 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:52:16.405513 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:52:16.406062 (+ 549us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:52:16.407703 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:52:16.409805 (+ 2102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:52:16.411386 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:52:16.412393 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:52:16.412625 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:52:16.417426 (+ 4801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:52:16.417437 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:52:16.417455 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:52:16.417506 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:52:16.422395 (+ 4889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:52:16.423210 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:52:16.423227 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:52:16.423256 (+ 29us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:52:16.423474 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:52:16.423973 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:52:16.423978 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:52:16.423987 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:52:16.424602 (+ 615us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:52:16.425053 (+ 451us) server_negotiation.cc:1035] Waiting for connection context
0807 10:52:16.425551 (+ 498us) server_negotiation.cc:299] Negotiation successful
0807 10:52:16.426072 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1521,"thread_start_us":1127,"threads_started":1}
I20260807 10:52:16.430454 20255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7843cf9c69554d34872bb8cfe65fd06a: No bootstrap required, opened a new log
I20260807 10:52:16.442128 20255 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7843cf9c69554d34872bb8cfe65fd06a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7843cf9c69554d34872bb8cfe65fd06a" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 35913 } }
I20260807 10:52:16.442705 20255 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7843cf9c69554d34872bb8cfe65fd06a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:52:16.442904 20255 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7843cf9c69554d34872bb8cfe65fd06a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7843cf9c69554d34872bb8cfe65fd06a, State: Initialized, Role: FOLLOWER
I20260807 10:52:16.443567 20255 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7843cf9c69554d34872bb8cfe65fd06a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7843cf9c69554d34872bb8cfe65fd06a" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 35913 } }
I20260807 10:52:16.443868 20255 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7843cf9c69554d34872bb8cfe65fd06a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:52:16.444072 20255 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7843cf9c69554d34872bb8cfe65fd06a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:52:16.444305 20255 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7843cf9c69554d34872bb8cfe65fd06a [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:52:16.447501 20255 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7843cf9c69554d34872bb8cfe65fd06a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7843cf9c69554d34872bb8cfe65fd06a" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 35913 } }
I20260807 10:52:16.448351 20255 leader_election.cc:302] T 00000000000000000000000000000000 P 7843cf9c69554d34872bb8cfe65fd06a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7843cf9c69554d34872bb8cfe65fd06a; no voters:
I20260807 10:52:16.449582 20255 leader_election.cc:288] T 00000000000000000000000000000000 P 7843cf9c69554d34872bb8cfe65fd06a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:52:16.449775 20262 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7843cf9c69554d34872bb8cfe65fd06a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:52:16.451620 20262 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7843cf9c69554d34872bb8cfe65fd06a [term 1 LEADER]: Becoming Leader. State: Replica: 7843cf9c69554d34872bb8cfe65fd06a, State: Running, Role: LEADER
I20260807 10:52:16.452579 20262 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7843cf9c69554d34872bb8cfe65fd06a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7843cf9c69554d34872bb8cfe65fd06a" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 35913 } }
I20260807 10:52:16.453018 20255 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7843cf9c69554d34872bb8cfe65fd06a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:52:16.457325 20264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7843cf9c69554d34872bb8cfe65fd06a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7843cf9c69554d34872bb8cfe65fd06a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7843cf9c69554d34872bb8cfe65fd06a" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 35913 } } }
I20260807 10:52:16.458343 20264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7843cf9c69554d34872bb8cfe65fd06a [sys.catalog]: This master's current role is: LEADER
I20260807 10:52:16.459558 20267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7843cf9c69554d34872bb8cfe65fd06a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7843cf9c69554d34872bb8cfe65fd06a. Latest consensus state: current_term: 1 leader_uuid: "7843cf9c69554d34872bb8cfe65fd06a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7843cf9c69554d34872bb8cfe65fd06a" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 35913 } } }
I20260807 10:52:16.460332 20267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7843cf9c69554d34872bb8cfe65fd06a [sys.catalog]: This master's current role is: LEADER
I20260807 10:52:16.462383 20272 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:52:16.471383 20272 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:52:16.481966 20255 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 10:52:16.494343 20272 catalog_manager.cc:1347] Generated new cluster ID: 248c914de4124c078d0dc2c035dfdbd1
I20260807 10:52:16.494992 20272 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:52:16.511587 20272 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:52:16.513767 20272 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:52:16.534438 20272 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7843cf9c69554d34872bb8cfe65fd06a: Generated new TSK 0
I20260807 10:52:16.535974 20272 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.73.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab.
Entry for principal kudu/127.3.73.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab.
I20260807 10:52:17.463022 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.73.212:59437/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.193:0
--local_ip_for_outbound_sockets=127.3.73.193
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:35913
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:44891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 10:52:17 dist-test-slave-x5fp krb5kdc[19461](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786099937, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 10:52:18.118723 20324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:52:18.119163 20324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:52:18.119427 20324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:52:18.130002 20324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:52:18.130296 20324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:52:18.130584 20324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.193
I20260807 10:52:18.144294 20324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:44891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.3.73.212:59437/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:35913
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.193
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:52:18.146112 20324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:52:18.147794 20324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:52:18.175673 20339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:52:18.176962 20342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:52:18.178656 20347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:52:18.181378 20324 server_base.cc:1034] running on GCE node
I20260807 10:52:18.182343 20324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:52:18.183820 20324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:52:18.185141 20324 hybrid_clock.cc:648] HybridClock initialized: now 1786099938185033 us; error 84 us; skew 500 ppm
I20260807 10:52:18.185670 20324 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:52:18 dist-test-slave-x5fp krb5kdc[19461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099938, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:52:18.191022 20324 init.cc:377] Logged in from keytab as kudu/127.3.73.193@KRBTEST.COM (short username kudu)
I20260807 10:52:18.197578 20324 webserver.cc:460] Webserver started at http://127.3.73.193:44167/ using document root <none> and password file <none>
I20260807 10:52:18.199021 20324 fs_manager.cc:359] Metadata directory not provided
I20260807 10:52:18.199322 20324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:52:18.199937 20324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 10:52:18 dist-test-slave-x5fp krb5kdc[19461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099938, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for HTTP/127.3.73.212@KRBTEST.COM
I20260807 10:52:18.225771 20324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-0/data/instance:
uuid: "a7665255ff3c481ca945de3da492cfcb"
format_stamp: "Formatted at 2026-08-07 10:52:18 on dist-test-slave-x5fp"
server_key: "99ca889d88dfd2d34077fc2e5b695bb4"
server_key_iv: "c11d6ab6956bd70a60f0d0821eedd10b"
server_key_version: "kuduclusterkey@0"
I20260807 10:52:18.227669 20324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance:
uuid: "a7665255ff3c481ca945de3da492cfcb"
format_stamp: "Formatted at 2026-08-07 10:52:18 on dist-test-slave-x5fp"
server_key: "99ca889d88dfd2d34077fc2e5b695bb4"
server_key_iv: "c11d6ab6956bd70a60f0d0821eedd10b"
server_key_version: "kuduclusterkey@0"
I20260807 10:52:18.238762 20324 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.004s sys 0.004s
I20260807 10:52:18.281358 20355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:52:18.283639 20324 fs_manager.cc:725] Time spent opening block manager: real 0.021s user 0.003s sys 0.016s
I20260807 10:52:18.283965 20324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-0/wal
uuid: "a7665255ff3c481ca945de3da492cfcb"
format_stamp: "Formatted at 2026-08-07 10:52:18 on dist-test-slave-x5fp"
server_key: "99ca889d88dfd2d34077fc2e5b695bb4"
server_key_iv: "c11d6ab6956bd70a60f0d0821eedd10b"
server_key_version: "kuduclusterkey@0"
I20260807 10:52:18.284338 20324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:52:18.377662 20324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:52:18.387261 20324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:52:18.390048 20324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:52:18.411810 20324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
Aug 07 10:52:18 dist-test-slave-x5fp krb5kdc[19461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099937, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for HTTP/127.3.73.212@KRBTEST.COM
I20260807 10:52:18.417737 20324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:52:18.417891 20324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:52:18.418149 20324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:52:18.418258 20324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 10:52:18 dist-test-slave-x5fp krb5kdc[19461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099937, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for HTTP/127.3.73.212@KRBTEST.COM
I20260807 10:52:19.008745 20324 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.193:34227
I20260807 10:52:19.008754 20520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.193:34227 every 8 connection(s)
I20260807 10:52:19.011627 20324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
I20260807 10:52:19.014106 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 20324
I20260807 10:52:19.014356 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance
I20260807 10:52:19.015759 3367 external_mini_cluster.cc:1428] Setting key b3e0a2b7a2f5f8f96a5dd6047143719e
WARNING: no policy specified for kudu/127.3.73.194@KRBTEST.COM; defaulting to no policy
Aug 07 10:52:19 dist-test-slave-x5fp krb5kdc[19461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099938, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.194@KRBTEST.COM" created.
I20260807 10:52:19.118227 20524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:52:19.032075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:39149 (local address 127.3.73.254:35913)
0807 10:52:19.047619 (+ 15544us) server_negotiation.cc:206] Beginning negotiation
0807 10:52:19.047627 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:52:19.059007 (+ 11380us) server_negotiation.cc:407] Connection header received
0807 10:52:19.072943 (+ 13936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:52:19.072951 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:52:19.073070 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:52:19.073230 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:52:19.078962 (+ 5732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:52:19.080482 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:52:19.084779 (+ 4297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:52:19.085805 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:52:19.085952 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:52:19.109634 (+ 23682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:52:19.109643 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:52:19.109648 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:52:19.109704 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:52:19.113595 (+ 3891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:52:19.114819 (+ 1224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:52:19.114826 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:52:19.114829 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:52:19.115004 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:52:19.115599 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:52:19.115604 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:52:19.115608 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:52:19.116074 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:52:19.116205 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 10:52:19.117457 (+ 1252us) server_negotiation.cc:299] Negotiation successful
0807 10:52:19.117728 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15337,"thread_start_us":15121,"threads_started":1}
I20260807 10:52:19.121362 20523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:52:19.031312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35913 (local address 127.3.73.193:39149)
0807 10:52:19.049006 (+ 17694us) negotiation.cc:107] Waiting for socket to connect
0807 10:52:19.049079 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0807 10:52:19.068599 (+ 19520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:52:19.074800 (+ 6201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:52:19.074816 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:52:19.075928 (+ 1112us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:52:19.078618 (+ 2690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:52:19.078661 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:52:19.080814 (+ 2153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:52:19.080820 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:52:19.083551 (+ 2731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:52:19.083574 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:52:19.086703 (+ 3129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:52:19.086715 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:52:19.087204 (+ 489us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:52:19.088395 (+ 1191us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:52:19.088434 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:52:19.095076 (+ 6642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:52:19.113863 (+ 18787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:52:19.113874 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:52:19.113898 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:52:19.114627 (+ 729us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:52:19.115164 (+ 537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:52:19.115169 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:52:19.115173 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:52:19.115457 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:52:19.116322 (+ 865us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:52:19.116331 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:52:19.116954 (+ 623us) client_negotiation.cc:769] Sending connection context
0807 10:52:19.118922 (+ 1968us) client_negotiation.cc:240] Negotiation successful
0807 10:52:19.119607 (+ 685us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17102,"thread_start_us":24250,"threads_started":1}
I20260807 10:52:19.124048 20521 heartbeater.cc:347] Connected to a master server at 127.3.73.254:35913
I20260807 10:52:19.124430 20521 heartbeater.cc:464] Registering TS with master...
I20260807 10:52:19.125555 20521 heartbeater.cc:511] Master 127.3.73.254:35913 requested a full tablet report, sending...
I20260807 10:52:19.128738 20206 ts_manager.cc:194] Registered new tserver with Master: a7665255ff3c481ca945de3da492cfcb (127.3.73.193:34227)
I20260807 10:52:19.131215 20206 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.193@KRBTEST.COM'} at 127.3.73.193:39149
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab.
Entry for principal kudu/127.3.73.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab.
W20260807 10:52:19.169013 3367 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.3.73.194: real 0.149s user 0.000s sys 0.031s
I20260807 10:52:19.170364 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.73.212:59437/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.194:0
--local_ip_for_outbound_sockets=127.3.73.194
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:35913
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:44891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:52:19.932857 20527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:52:19.933514 20527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:52:19.933818 20527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:52:19.945772 20527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:52:19.946219 20527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:52:19.946530 20527 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.194
I20260807 10:52:19.966172 20527 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:44891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.3.73.212:59437/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:35913
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.194
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:52:19.968576 20527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:52:19.970990 20527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:52:19.994534 20550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:52:19.994753 20545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:52:19.994859 20546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:52:19.997210 20527 server_base.cc:1034] running on GCE node
I20260807 10:52:19.998256 20527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:52:19.999778 20527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:52:20.001269 20527 hybrid_clock.cc:648] HybridClock initialized: now 1786099940001101 us; error 78 us; skew 500 ppm
I20260807 10:52:20.001838 20527 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:52:20 dist-test-slave-x5fp krb5kdc[19461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099940, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:52:20.007762 20527 init.cc:377] Logged in from keytab as kudu/127.3.73.194@KRBTEST.COM (short username kudu)
I20260807 10:52:20.010424 20527 webserver.cc:460] Webserver started at http://127.3.73.194:33937/ using document root <none> and password file <none>
I20260807 10:52:20.011739 20527 fs_manager.cc:359] Metadata directory not provided
I20260807 10:52:20.011969 20527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:52:20.012598 20527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 10:52:20 dist-test-slave-x5fp krb5kdc[19461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099940, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for HTTP/127.3.73.212@KRBTEST.COM
I20260807 10:52:20.035487 20527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-1/data/instance:
uuid: "79b69651e30f46a3858cbafce82e2824"
format_stamp: "Formatted at 2026-08-07 10:52:20 on dist-test-slave-x5fp"
server_key: "f18a5d802edb1e608ace07cf115b614a"
server_key_iv: "908824584223e538e0f47de340225845"
server_key_version: "kuduclusterkey@0"
I20260807 10:52:20.037256 20527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance:
uuid: "79b69651e30f46a3858cbafce82e2824"
format_stamp: "Formatted at 2026-08-07 10:52:20 on dist-test-slave-x5fp"
server_key: "f18a5d802edb1e608ace07cf115b614a"
server_key_iv: "908824584223e538e0f47de340225845"
server_key_version: "kuduclusterkey@0"
I20260807 10:52:20.048022 20527 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.000s sys 0.009s
I20260807 10:52:20.070755 20561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:52:20.074072 20527 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.005s sys 0.000s
I20260807 10:52:20.074455 20527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-1/wal
uuid: "79b69651e30f46a3858cbafce82e2824"
format_stamp: "Formatted at 2026-08-07 10:52:20 on dist-test-slave-x5fp"
server_key: "f18a5d802edb1e608ace07cf115b614a"
server_key_iv: "908824584223e538e0f47de340225845"
server_key_version: "kuduclusterkey@0"
I20260807 10:52:20.074852 20527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:52:20.135782 20521 heartbeater.cc:503] Master 127.3.73.254:35913 was elected leader, sending a full tablet report...
I20260807 10:52:20.148414 20527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:52:20.157027 20527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:52:20.157721 20527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:52:20.160096 20527 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:52:20.163357 20527 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:52:20.163561 20527 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:52:20.163818 20527 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:52:20.163957 20527 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:52:20.269073 20527 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.194:43237
I20260807 10:52:20.269229 20751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.194:43237 every 8 connection(s)
I20260807 10:52:20.271461 20527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
I20260807 10:52:20.278596 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 20527
I20260807 10:52:20.278846 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance
I20260807 10:52:20.279902 3367 external_mini_cluster.cc:1428] Setting key dba077aa04f1344aa0e42de53b714b60
Aug 07 10:52:20 dist-test-slave-x5fp krb5kdc[19461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099940, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:52:20.317739 20756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:52:20.278045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35913 (local address 127.3.73.194:35195)
0807 10:52:20.280255 (+ 2210us) negotiation.cc:107] Waiting for socket to connect
0807 10:52:20.280329 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0807 10:52:20.284584 (+ 4255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:52:20.285740 (+ 1156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:52:20.285759 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:52:20.287358 (+ 1599us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:52:20.288571 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:52:20.288606 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:52:20.290843 (+ 2237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:52:20.290852 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:52:20.294222 (+ 3370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:52:20.294246 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:52:20.295859 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:52:20.295873 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:52:20.296281 (+ 408us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:52:20.297489 (+ 1208us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:52:20.297565 (+ 76us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:52:20.301497 (+ 3932us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:52:20.309054 (+ 7557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:52:20.309067 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:52:20.309102 (+ 35us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:52:20.310438 (+ 1336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:52:20.311339 (+ 901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:52:20.311348 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:52:20.311355 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:52:20.311923 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:52:20.313118 (+ 1195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:52:20.313129 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:52:20.313861 (+ 732us) client_negotiation.cc:769] Sending connection context
0807 10:52:20.314310 (+ 449us) client_negotiation.cc:240] Negotiation successful
0807 10:52:20.315255 (+ 945us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1728,"thread_start_us":1256,"threads_started":1}
I20260807 10:52:20.320436 20755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:52:20.277100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:35195 (local address 127.3.73.254:35913)
0807 10:52:20.278579 (+ 1479us) server_negotiation.cc:206] Beginning negotiation
0807 10:52:20.278589 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:52:20.280533 (+ 1944us) server_negotiation.cc:407] Connection header received
0807 10:52:20.285033 (+ 4500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:52:20.285039 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:52:20.285129 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:52:20.285298 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:52:20.288872 (+ 3574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:52:20.290559 (+ 1687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:52:20.294439 (+ 3880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:52:20.295342 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:52:20.295489 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:52:20.301860 (+ 6371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:52:20.301869 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:52:20.301874 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:52:20.302001 (+ 127us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:52:20.308646 (+ 6645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:52:20.310788 (+ 2142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:52:20.310798 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:52:20.310805 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:52:20.311068 (+ 263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:52:20.312140 (+ 1072us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:52:20.312148 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:52:20.312155 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:52:20.312891 (+ 736us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:52:20.319015 (+ 6124us) server_negotiation.cc:1035] Waiting for connection context
0807 10:52:20.319300 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 10:52:20.319715 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1245,"thread_start_us":1504,"threads_started":1}
I20260807 10:52:20.322677 20752 heartbeater.cc:347] Connected to a master server at 127.3.73.254:35913
I20260807 10:52:20.323210 20752 heartbeater.cc:464] Registering TS with master...
I20260807 10:52:20.324222 20752 heartbeater.cc:511] Master 127.3.73.254:35913 requested a full tablet report, sending...
I20260807 10:52:20.326397 20206 ts_manager.cc:194] Registered new tserver with Master: 79b69651e30f46a3858cbafce82e2824 (127.3.73.194:43237)
WARNING: no policy specified for kudu/127.3.73.195@KRBTEST.COM; defaulting to no policy
I20260807 10:52:20.327445 20206 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.194@KRBTEST.COM'} at 127.3.73.194:35195
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab.
Entry for principal kudu/127.3.73.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab.
I20260807 10:52:20.381486 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.73.212:59437/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.195:0
--local_ip_for_outbound_sockets=127.3.73.195
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:35913
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:44891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:52:21.107757 20760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:52:21.108232 20760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:52:21.108536 20760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:52:21.119485 20760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:52:21.119796 20760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:52:21.120111 20760 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.195
I20260807 10:52:21.133450 20760 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:44891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.3.73.212:59437/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:35913
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.195
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:52:21.135028 20760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:52:21.136653 20760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:52:21.156962 20765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:52:21.157821 20766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:52:21.158996 20769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:52:21.159222 20760 server_base.cc:1034] running on GCE node
I20260807 10:52:21.160153 20760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:52:21.161531 20760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:52:21.162832 20760 hybrid_clock.cc:648] HybridClock initialized: now 1786099941162771 us; error 48 us; skew 500 ppm
I20260807 10:52:21.163308 20760 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:52:21 dist-test-slave-x5fp krb5kdc[19461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099941, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:52:21.175079 20760 init.cc:377] Logged in from keytab as kudu/127.3.73.195@KRBTEST.COM (short username kudu)
I20260807 10:52:21.177588 20760 webserver.cc:460] Webserver started at http://127.3.73.195:40657/ using document root <none> and password file <none>
I20260807 10:52:21.178568 20760 fs_manager.cc:359] Metadata directory not provided
I20260807 10:52:21.178892 20760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:52:21.179447 20760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 10:52:21 dist-test-slave-x5fp krb5kdc[19461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099941, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for HTTP/127.3.73.212@KRBTEST.COM
I20260807 10:52:21.201792 20760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-2/data/instance:
uuid: "2207ae398abe4437b889acb0a1c46191"
format_stamp: "Formatted at 2026-08-07 10:52:21 on dist-test-slave-x5fp"
server_key: "7ff4d6d131f6bf2b5e00c59cc245ff0c"
server_key_iv: "64c9ec5ec6f7bcc9f4655d6d84e5afd4"
server_key_version: "kuduclusterkey@0"
I20260807 10:52:21.203116 20760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance:
uuid: "2207ae398abe4437b889acb0a1c46191"
format_stamp: "Formatted at 2026-08-07 10:52:21 on dist-test-slave-x5fp"
server_key: "7ff4d6d131f6bf2b5e00c59cc245ff0c"
server_key_iv: "64c9ec5ec6f7bcc9f4655d6d84e5afd4"
server_key_version: "kuduclusterkey@0"
I20260807 10:52:21.211755 20760 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.000s
I20260807 10:52:21.234015 20780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:52:21.236239 20760 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260807 10:52:21.236588 20760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-2/wal
uuid: "2207ae398abe4437b889acb0a1c46191"
format_stamp: "Formatted at 2026-08-07 10:52:21 on dist-test-slave-x5fp"
server_key: "7ff4d6d131f6bf2b5e00c59cc245ff0c"
server_key_iv: "64c9ec5ec6f7bcc9f4655d6d84e5afd4"
server_key_version: "kuduclusterkey@0"
I20260807 10:52:21.237111 20760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:52:21.257567 20760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:52:21.263741 20760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:52:21.264436 20760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:52:21.266610 20760 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:52:21.269897 20760 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:52:21.270180 20760 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:52:21.270428 20760 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:52:21.270541 20760 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:52:21.331871 20752 heartbeater.cc:503] Master 127.3.73.254:35913 was elected leader, sending a full tablet report...
I20260807 10:52:21.348078 20760 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.195:40687
I20260807 10:52:21.348129 20948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.195:40687 every 8 connection(s)
I20260807 10:52:21.349961 20760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
I20260807 10:52:21.353698 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 20760
I20260807 10:52:21.353914 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance
I20260807 10:52:21.354688 3367 external_mini_cluster.cc:1428] Setting key 55defcfb1bdc9501742aefb6e86fd526
Aug 07 10:52:21 dist-test-slave-x5fp krb5kdc[19461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099941, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:52:21.383152 20952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:52:21.354131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:47375 (local address 127.3.73.254:35913)
0807 10:52:21.354974 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0807 10:52:21.354982 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:52:21.356781 (+ 1799us) server_negotiation.cc:407] Connection header received
0807 10:52:21.359363 (+ 2582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:52:21.359368 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:52:21.359500 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:52:21.359635 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:52:21.362450 (+ 2815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:52:21.364521 (+ 2071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:52:21.367511 (+ 2990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:52:21.368662 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:52:21.368969 (+ 307us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:52:21.374251 (+ 5282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:52:21.374257 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:52:21.374261 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:52:21.374308 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:52:21.378449 (+ 4141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:52:21.379616 (+ 1167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:52:21.379626 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:52:21.379631 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:52:21.379839 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:52:21.380487 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:52:21.380492 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:52:21.380494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:52:21.380961 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:52:21.381171 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0807 10:52:21.382217 (+ 1046us) server_negotiation.cc:299] Negotiation successful
0807 10:52:21.382549 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":401,"threads_started":1}
I20260807 10:52:21.384806 20953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:52:21.355044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35913 (local address 127.3.73.195:47375)
0807 10:52:21.356544 (+ 1500us) negotiation.cc:107] Waiting for socket to connect
0807 10:52:21.356618 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0807 10:52:21.359044 (+ 2426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:52:21.359971 (+ 927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:52:21.359981 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:52:21.360697 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:52:21.362193 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:52:21.362229 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:52:21.364896 (+ 2667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:52:21.364901 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:52:21.367177 (+ 2276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:52:21.367196 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:52:21.369087 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:52:21.369094 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:52:21.369435 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:52:21.370536 (+ 1101us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:52:21.370576 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:52:21.373820 (+ 3244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:52:21.378652 (+ 4832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:52:21.378664 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:52:21.378695 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:52:21.379431 (+ 736us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:52:21.379985 (+ 554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:52:21.379991 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:52:21.379995 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:52:21.380330 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:52:21.381123 (+ 793us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:52:21.381134 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:52:21.381689 (+ 555us) client_negotiation.cc:769] Sending connection context
0807 10:52:21.382133 (+ 444us) client_negotiation.cc:240] Negotiation successful
0807 10:52:21.382963 (+ 830us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1118,"thread_start_us":735,"threads_started":1}
I20260807 10:52:21.387276 20949 heartbeater.cc:347] Connected to a master server at 127.3.73.254:35913
I20260807 10:52:21.387769 20949 heartbeater.cc:464] Registering TS with master...
I20260807 10:52:21.389371 20949 heartbeater.cc:511] Master 127.3.73.254:35913 requested a full tablet report, sending...
I20260807 10:52:21.391944 20206 ts_manager.cc:194] Registered new tserver with Master: 2207ae398abe4437b889acb0a1c46191 (127.3.73.195:40687)
I20260807 10:52:21.393230 20206 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.195@KRBTEST.COM'} at 127.3.73.195:47375
I20260807 10:52:21.404171 3367 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 10:52:21.431875 20952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:52:21.412827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47302 (local address 127.3.73.254:35913)
0807 10:52:21.413083 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0807 10:52:21.413091 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:52:21.414184 (+ 1093us) server_negotiation.cc:407] Connection header received
0807 10:52:21.414391 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:52:21.414396 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:52:21.414530 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:52:21.414633 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:52:21.416630 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:52:21.417990 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:52:21.419940 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:52:21.420789 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:52:21.420981 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:52:21.424188 (+ 3207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:52:21.424194 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:52:21.424197 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:52:21.424237 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:52:21.428319 (+ 4082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:52:21.429341 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:52:21.429346 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:52:21.429348 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:52:21.429550 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:52:21.429955 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:52:21.429958 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:52:21.429960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:52:21.430374 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:52:21.430605 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 10:52:21.430995 (+ 390us) server_negotiation.cc:299] Negotiation successful
0807 10:52:21.431308 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 10:52:21.437696 20206 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47302:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 10:52:21.449630 20206 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 10:52:21.495750 20966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:52:21.471169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:34227 (local address 127.0.0.1:40356)
0807 10:52:21.479132 (+ 7963us) negotiation.cc:107] Waiting for socket to connect
0807 10:52:21.479203 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0807 10:52:21.479486 (+ 283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:52:21.484005 (+ 4519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:52:21.484017 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:52:21.484063 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:52:21.484953 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:52:21.484974 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:52:21.488165 (+ 3191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:52:21.488172 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:52:21.492298 (+ 4126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:52:21.492329 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:52:21.494329 (+ 2000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:52:21.494337 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:52:21.494708 (+ 371us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:52:21.494785 (+ 77us) client_negotiation.cc:769] Sending connection context
0807 10:52:21.494917 (+ 132us) client_negotiation.cc:240] Negotiation successful
0807 10:52:21.495127 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7519,"mutex_wait_us":119,"thread_start_us":6965,"threads_started":1}
I20260807 10:52:21.496995 20968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:52:21.471378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40356 (local address 127.3.73.193:34227)
0807 10:52:21.483242 (+ 11864us) server_negotiation.cc:206] Beginning negotiation
0807 10:52:21.483253 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 10:52:21.483286 (+ 33us) server_negotiation.cc:407] Connection header received
0807 10:52:21.483401 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:52:21.483412 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:52:21.483741 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:52:21.483984 (+ 243us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:52:21.485355 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:52:21.488003 (+ 2648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:52:21.492554 (+ 4551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:52:21.494008 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:52:21.494196 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:52:21.494530 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0807 10:52:21.496344 (+ 1814us) server_negotiation.cc:299] Negotiation successful
0807 10:52:21.496570 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11649,"thread_start_us":12299,"threads_started":1}
I20260807 10:52:21.498898 20970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:52:21.472675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58614 (local address 127.3.73.195:40687)
0807 10:52:21.488255 (+ 15580us) server_negotiation.cc:206] Beginning negotiation
0807 10:52:21.488265 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:52:21.488312 (+ 47us) server_negotiation.cc:407] Connection header received
0807 10:52:21.488410 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:52:21.488417 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:52:21.488672 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:52:21.489669 (+ 997us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:52:21.490540 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:52:21.491916 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:52:21.495869 (+ 3953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:52:21.497319 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:52:21.497514 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:52:21.497791 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0807 10:52:21.498175 (+ 384us) server_negotiation.cc:299] Negotiation successful
0807 10:52:21.498365 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15284,"thread_start_us":14610,"threads_started":1}
I20260807 10:52:21.501211 20971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:52:21.471800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:40687 (local address 127.0.0.1:58614)
0807 10:52:21.485530 (+ 13730us) negotiation.cc:107] Waiting for socket to connect
0807 10:52:21.485557 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0807 10:52:21.485723 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:52:21.489005 (+ 3282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:52:21.489010 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:52:21.489036 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:52:21.490332 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:52:21.490345 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:52:21.492111 (+ 1766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:52:21.492116 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:52:21.495657 (+ 3541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:52:21.495675 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:52:21.497721 (+ 2046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:52:21.497730 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:52:21.497975 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:52:21.498008 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 10:52:21.498106 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 10:52:21.498258 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13555,"mutex_wait_us":57,"thread_start_us":14792,"threads_started":1}
I20260807 10:52:21.501341 20422 tablet_service.cc:1467] Processing CreateTablet for tablet 2837431645b941db88584af86ccb707c (DEFAULT_TABLE table=test-table [id=31397f87b2b94e2dbda5b17c74417104]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:52:21.502257 20969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:52:21.479504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:43237 (local address 127.0.0.1:37268)
0807 10:52:21.487039 (+ 7535us) negotiation.cc:107] Waiting for socket to connect
0807 10:52:21.487061 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 10:52:21.487234 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:52:21.488493 (+ 1259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:52:21.488498 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:52:21.488522 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:52:21.489442 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:52:21.489458 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:52:21.492488 (+ 3030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:52:21.492495 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:52:21.496974 (+ 4479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:52:21.496993 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:52:21.501004 (+ 4011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:52:21.501011 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:52:21.501253 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:52:21.501301 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 10:52:21.501394 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 10:52:21.501546 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7387,"mutex_wait_us":1672,"thread_start_us":6362,"threads_started":1}
I20260807 10:52:21.503438 20422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2837431645b941db88584af86ccb707c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:52:21.503455 20972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:52:21.480129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37268 (local address 127.3.73.194:43237)
0807 10:52:21.485002 (+ 4873us) server_negotiation.cc:206] Beginning negotiation
0807 10:52:21.485011 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:52:21.487424 (+ 2413us) server_negotiation.cc:407] Connection header received
0807 10:52:21.487570 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:52:21.487579 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:52:21.487927 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:52:21.488161 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:52:21.489919 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:52:21.492271 (+ 2352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:52:21.497286 (+ 5015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:52:21.499138 (+ 1852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:52:21.499275 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:52:21.499463 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 10:52:21.501515 (+ 2052us) server_negotiation.cc:299] Negotiation successful
0807 10:52:21.503048 (+ 1533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4642,"thread_start_us":4396,"threads_started":1}
I20260807 10:52:21.505676 20855 tablet_service.cc:1467] Processing CreateTablet for tablet 2837431645b941db88584af86ccb707c (DEFAULT_TABLE table=test-table [id=31397f87b2b94e2dbda5b17c74417104]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:52:21.506847 20647 tablet_service.cc:1467] Processing CreateTablet for tablet 2837431645b941db88584af86ccb707c (DEFAULT_TABLE table=test-table [id=31397f87b2b94e2dbda5b17c74417104]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:52:21.507709 20855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2837431645b941db88584af86ccb707c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:52:21.508553 20647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2837431645b941db88584af86ccb707c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:52:21.523917 20974 tablet_bootstrap.cc:492] T 2837431645b941db88584af86ccb707c P 79b69651e30f46a3858cbafce82e2824: Bootstrap starting.
I20260807 10:52:21.525717 20976 tablet_bootstrap.cc:492] T 2837431645b941db88584af86ccb707c P a7665255ff3c481ca945de3da492cfcb: Bootstrap starting.
I20260807 10:52:21.529470 20977 tablet_bootstrap.cc:492] T 2837431645b941db88584af86ccb707c P 2207ae398abe4437b889acb0a1c46191: Bootstrap starting.
I20260807 10:52:21.532905 20974 tablet_bootstrap.cc:654] T 2837431645b941db88584af86ccb707c P 79b69651e30f46a3858cbafce82e2824: Neither blocks nor log segments found. Creating new log.
I20260807 10:52:21.533463 20976 tablet_bootstrap.cc:654] T 2837431645b941db88584af86ccb707c P a7665255ff3c481ca945de3da492cfcb: Neither blocks nor log segments found. Creating new log.
I20260807 10:52:21.535077 20977 tablet_bootstrap.cc:654] T 2837431645b941db88584af86ccb707c P 2207ae398abe4437b889acb0a1c46191: Neither blocks nor log segments found. Creating new log.
I20260807 10:52:21.536407 20976 log.cc:824] T 2837431645b941db88584af86ccb707c P a7665255ff3c481ca945de3da492cfcb: Log is configured to *not* fsync() on all Append() calls
I20260807 10:52:21.536374 20974 log.cc:824] T 2837431645b941db88584af86ccb707c P 79b69651e30f46a3858cbafce82e2824: Log is configured to *not* fsync() on all Append() calls
I20260807 10:52:21.537673 20977 log.cc:824] T 2837431645b941db88584af86ccb707c P 2207ae398abe4437b889acb0a1c46191: Log is configured to *not* fsync() on all Append() calls
I20260807 10:52:21.542662 20976 tablet_bootstrap.cc:492] T 2837431645b941db88584af86ccb707c P a7665255ff3c481ca945de3da492cfcb: No bootstrap required, opened a new log
I20260807 10:52:21.543327 20976 ts_tablet_manager.cc:1397] T 2837431645b941db88584af86ccb707c P a7665255ff3c481ca945de3da492cfcb: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.011s
I20260807 10:52:21.544008 20977 tablet_bootstrap.cc:492] T 2837431645b941db88584af86ccb707c P 2207ae398abe4437b889acb0a1c46191: No bootstrap required, opened a new log
I20260807 10:52:21.544459 20974 tablet_bootstrap.cc:492] T 2837431645b941db88584af86ccb707c P 79b69651e30f46a3858cbafce82e2824: No bootstrap required, opened a new log
I20260807 10:52:21.544528 20977 ts_tablet_manager.cc:1397] T 2837431645b941db88584af86ccb707c P 2207ae398abe4437b889acb0a1c46191: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.009s
I20260807 10:52:21.545014 20974 ts_tablet_manager.cc:1397] T 2837431645b941db88584af86ccb707c P 79b69651e30f46a3858cbafce82e2824: Time spent bootstrapping tablet: real 0.021s user 0.016s sys 0.004s
I20260807 10:52:21.555441 20977 raft_consensus.cc:348] T 2837431645b941db88584af86ccb707c P 2207ae398abe4437b889acb0a1c46191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7665255ff3c481ca945de3da492cfcb" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 34227 } } peers { permanent_uuid: "79b69651e30f46a3858cbafce82e2824" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43237 } } peers { permanent_uuid: "2207ae398abe4437b889acb0a1c46191" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 40687 } }
I20260807 10:52:21.556114 20977 raft_consensus.cc:374] T 2837431645b941db88584af86ccb707c P 2207ae398abe4437b889acb0a1c46191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:52:21.556375 20977 raft_consensus.cc:729] T 2837431645b941db88584af86ccb707c P 2207ae398abe4437b889acb0a1c46191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2207ae398abe4437b889acb0a1c46191, State: Initialized, Role: FOLLOWER
I20260807 10:52:21.557428 20977 consensus_queue.cc:260] T 2837431645b941db88584af86ccb707c P 2207ae398abe4437b889acb0a1c46191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7665255ff3c481ca945de3da492cfcb" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 34227 } } peers { permanent_uuid: "79b69651e30f46a3858cbafce82e2824" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43237 } } peers { permanent_uuid: "2207ae398abe4437b889acb0a1c46191" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 40687 } }
I20260807 10:52:21.559473 20949 heartbeater.cc:503] Master 127.3.73.254:35913 was elected leader, sending a full tablet report...
I20260807 10:52:21.560449 20977 ts_tablet_manager.cc:1428] T 2837431645b941db88584af86ccb707c P 2207ae398abe4437b889acb0a1c46191: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
I20260807 10:52:21.560664 20976 raft_consensus.cc:348] T 2837431645b941db88584af86ccb707c P a7665255ff3c481ca945de3da492cfcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7665255ff3c481ca945de3da492cfcb" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 34227 } } peers { permanent_uuid: "79b69651e30f46a3858cbafce82e2824" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43237 } } peers { permanent_uuid: "2207ae398abe4437b889acb0a1c46191" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 40687 } }
I20260807 10:52:21.561307 20976 raft_consensus.cc:374] T 2837431645b941db88584af86ccb707c P a7665255ff3c481ca945de3da492cfcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:52:21.561451 20976 raft_consensus.cc:729] T 2837431645b941db88584af86ccb707c P a7665255ff3c481ca945de3da492cfcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7665255ff3c481ca945de3da492cfcb, State: Initialized, Role: FOLLOWER
I20260807 10:52:21.562215 20976 consensus_queue.cc:260] T 2837431645b941db88584af86ccb707c P a7665255ff3c481ca945de3da492cfcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7665255ff3c481ca945de3da492cfcb" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 34227 } } peers { permanent_uuid: "79b69651e30f46a3858cbafce82e2824" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43237 } } peers { permanent_uuid: "2207ae398abe4437b889acb0a1c46191" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 40687 } }
I20260807 10:52:21.563071 20974 raft_consensus.cc:348] T 2837431645b941db88584af86ccb707c P 79b69651e30f46a3858cbafce82e2824 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7665255ff3c481ca945de3da492cfcb" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 34227 } } peers { permanent_uuid: "79b69651e30f46a3858cbafce82e2824" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43237 } } peers { permanent_uuid: "2207ae398abe4437b889acb0a1c46191" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 40687 } }
I20260807 10:52:21.563953 20974 raft_consensus.cc:374] T 2837431645b941db88584af86ccb707c P 79b69651e30f46a3858cbafce82e2824 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:52:21.564168 20974 raft_consensus.cc:729] T 2837431645b941db88584af86ccb707c P 79b69651e30f46a3858cbafce82e2824 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79b69651e30f46a3858cbafce82e2824, State: Initialized, Role: FOLLOWER
I20260807 10:52:21.565147 20976 ts_tablet_manager.cc:1428] T 2837431645b941db88584af86ccb707c P a7665255ff3c481ca945de3da492cfcb: Time spent starting tablet: real 0.021s user 0.013s sys 0.008s
I20260807 10:52:21.565001 20974 consensus_queue.cc:260] T 2837431645b941db88584af86ccb707c P 79b69651e30f46a3858cbafce82e2824 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7665255ff3c481ca945de3da492cfcb" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 34227 } } peers { permanent_uuid: "79b69651e30f46a3858cbafce82e2824" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43237 } } peers { permanent_uuid: "2207ae398abe4437b889acb0a1c46191" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 40687 } }
I20260807 10:52:21.568387 20974 ts_tablet_manager.cc:1428] T 2837431645b941db88584af86ccb707c P 79b69651e30f46a3858cbafce82e2824: Time spent starting tablet: real 0.023s user 0.013s sys 0.008s
W20260807 10:52:21.603108 20951 tablet.cc:2369] T 2837431645b941db88584af86ccb707c P 2207ae398abe4437b889acb0a1c46191: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 10:52:21.776714 20753 tablet.cc:2369] T 2837431645b941db88584af86ccb707c P 79b69651e30f46a3858cbafce82e2824: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 10:52:21.794754 20522 tablet.cc:2369] T 2837431645b941db88584af86ccb707c P a7665255ff3c481ca945de3da492cfcb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:52:21.973058 20987 raft_consensus.cc:482] T 2837431645b941db88584af86ccb707c P 2207ae398abe4437b889acb0a1c46191 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 10:52:21.973374 20987 raft_consensus.cc:504] T 2837431645b941db88584af86ccb707c P 2207ae398abe4437b889acb0a1c46191 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7665255ff3c481ca945de3da492cfcb" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 34227 } } peers { permanent_uuid: "79b69651e30f46a3858cbafce82e2824" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43237 } } peers { permanent_uuid: "2207ae398abe4437b889acb0a1c46191" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 40687 } }
I20260807 10:52:21.975584 20987 leader_election.cc:288] T 2837431645b941db88584af86ccb707c P 2207ae398abe4437b889acb0a1c46191 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a7665255ff3c481ca945de3da492cfcb (127.3.73.193:34227), 79b69651e30f46a3858cbafce82e2824 (127.3.73.194:43237)
I20260807 10:52:21.985008 20972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:52:21.978061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:35069 (local address 127.3.73.194:43237)
0807 10:52:21.978251 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 10:52:21.978256 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:52:21.978987 (+ 731us) server_negotiation.cc:407] Connection header received
0807 10:52:21.979088 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:52:21.979092 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:52:21.979167 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:52:21.979251 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:52:21.979924 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:52:21.980665 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:52:21.982983 (+ 2318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:52:21.983891 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:52:21.984074 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:52:21.984165 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 10:52:21.984585 (+ 420us) server_negotiation.cc:299] Negotiation successful
0807 10:52:21.984742 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 10:52:21.985001 20968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:52:21.976350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:34997 (local address 127.3.73.193:34227)
0807 10:52:21.976538 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 10:52:21.976543 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:52:21.977481 (+ 938us) server_negotiation.cc:407] Connection header received
0807 10:52:21.977793 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:52:21.977799 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:52:21.977872 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:52:21.977975 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:52:21.979037 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:52:21.980252 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:52:21.982743 (+ 2491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:52:21.983844 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:52:21.984082 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:52:21.984165 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 10:52:21.984427 (+ 262us) server_negotiation.cc:299] Negotiation successful
0807 10:52:21.984565 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 10:52:21.985342 20992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:52:21.975856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:34227 (local address 127.3.73.195:34997)
0807 10:52:21.977328 (+ 1472us) negotiation.cc:107] Waiting for socket to connect
0807 10:52:21.977346 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 10:52:21.977633 (+ 287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:52:21.977996 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:52:21.978000 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:52:21.978026 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:52:21.978769 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:52:21.978785 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:52:21.980409 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:52:21.980415 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:52:21.982542 (+ 2127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:52:21.982556 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:52:21.984026 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:52:21.984033 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:52:21.984249 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:52:21.984274 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 10:52:21.984408 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 10:52:21.984559 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1353,"mutex_wait_us":1,"thread_start_us":1211,"threads_started":1}
I20260807 10:52:21.985484 20994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:52:21.977489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:43237 (local address 127.3.73.195:35069)
0807 10:52:21.978816 (+ 1327us) negotiation.cc:107] Waiting for socket to connect
0807 10:52:21.978839 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 10:52:21.978976 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:52:21.979297 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:52:21.979302 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:52:21.979329 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:52:21.979747 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:52:21.979760 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:52:21.980874 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:52:21.980879 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:52:21.982787 (+ 1908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:52:21.982800 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:52:21.984159 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:52:21.984166 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:52:21.984417 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:52:21.984442 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 10:52:21.984555 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 10:52:21.984689 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1188,"mutex_wait_us":4,"thread_start_us":837,"threads_started":1}
I20260807 10:52:21.986675 20676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2837431645b941db88584af86ccb707c" candidate_uuid: "2207ae398abe4437b889acb0a1c46191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79b69651e30f46a3858cbafce82e2824" is_pre_election: true
I20260807 10:52:21.986775 20450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2837431645b941db88584af86ccb707c" candidate_uuid: "2207ae398abe4437b889acb0a1c46191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7665255ff3c481ca945de3da492cfcb" is_pre_election: true
I20260807 10:52:21.987354 20676 raft_consensus.cc:2454] T 2837431645b941db88584af86ccb707c P 79b69651e30f46a3858cbafce82e2824 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2207ae398abe4437b889acb0a1c46191 in term 0.
I20260807 10:52:21.987347 20450 raft_consensus.cc:2454] T 2837431645b941db88584af86ccb707c P a7665255ff3c481ca945de3da492cfcb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2207ae398abe4437b889acb0a1c46191 in term 0.
I20260807 10:52:21.988124 20782 leader_election.cc:302] T 2837431645b941db88584af86ccb707c P 2207ae398abe4437b889acb0a1c46191 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2207ae398abe4437b889acb0a1c46191, a7665255ff3c481ca945de3da492cfcb; no voters:
I20260807 10:52:21.988636 20987 raft_consensus.cc:2781] T 2837431645b941db88584af86ccb707c P 2207ae398abe4437b889acb0a1c46191 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 10:52:21.988893 20987 raft_consensus.cc:482] T 2837431645b941db88584af86ccb707c P 2207ae398abe4437b889acb0a1c46191 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 10:52:21.989073 20987 raft_consensus.cc:3037] T 2837431645b941db88584af86ccb707c P 2207ae398abe4437b889acb0a1c46191 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:52:21.992192 20987 raft_consensus.cc:504] T 2837431645b941db88584af86ccb707c P 2207ae398abe4437b889acb0a1c46191 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7665255ff3c481ca945de3da492cfcb" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 34227 } } peers { permanent_uuid: "79b69651e30f46a3858cbafce82e2824" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43237 } } peers { permanent_uuid: "2207ae398abe4437b889acb0a1c46191" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 40687 } }
I20260807 10:52:21.993049 20987 leader_election.cc:288] T 2837431645b941db88584af86ccb707c P 2207ae398abe4437b889acb0a1c46191 [CANDIDATE]: Term 1 election: Requested vote from peers a7665255ff3c481ca945de3da492cfcb (127.3.73.193:34227), 79b69651e30f46a3858cbafce82e2824 (127.3.73.194:43237)
I20260807 10:52:21.993741 20450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2837431645b941db88584af86ccb707c" candidate_uuid: "2207ae398abe4437b889acb0a1c46191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a7665255ff3c481ca945de3da492cfcb"
I20260807 10:52:21.993912 20676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2837431645b941db88584af86ccb707c" candidate_uuid: "2207ae398abe4437b889acb0a1c46191" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79b69651e30f46a3858cbafce82e2824"
I20260807 10:52:21.994029 20450 raft_consensus.cc:3037] T 2837431645b941db88584af86ccb707c P a7665255ff3c481ca945de3da492cfcb [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:52:21.994196 20676 raft_consensus.cc:3037] T 2837431645b941db88584af86ccb707c P 79b69651e30f46a3858cbafce82e2824 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:52:21.996600 20676 raft_consensus.cc:2454] T 2837431645b941db88584af86ccb707c P 79b69651e30f46a3858cbafce82e2824 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2207ae398abe4437b889acb0a1c46191 in term 1.
I20260807 10:52:21.997126 20782 leader_election.cc:302] T 2837431645b941db88584af86ccb707c P 2207ae398abe4437b889acb0a1c46191 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2207ae398abe4437b889acb0a1c46191, 79b69651e30f46a3858cbafce82e2824; no voters:
I20260807 10:52:21.997488 20987 raft_consensus.cc:2781] T 2837431645b941db88584af86ccb707c P 2207ae398abe4437b889acb0a1c46191 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:52:21.997661 20450 raft_consensus.cc:2454] T 2837431645b941db88584af86ccb707c P a7665255ff3c481ca945de3da492cfcb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2207ae398abe4437b889acb0a1c46191 in term 1.
I20260807 10:52:21.998102 20987 raft_consensus.cc:686] T 2837431645b941db88584af86ccb707c P 2207ae398abe4437b889acb0a1c46191 [term 1 LEADER]: Becoming Leader. State: Replica: 2207ae398abe4437b889acb0a1c46191, State: Running, Role: LEADER
I20260807 10:52:21.998718 20987 consensus_queue.cc:237] T 2837431645b941db88584af86ccb707c P 2207ae398abe4437b889acb0a1c46191 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7665255ff3c481ca945de3da492cfcb" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 34227 } } peers { permanent_uuid: "79b69651e30f46a3858cbafce82e2824" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43237 } } peers { permanent_uuid: "2207ae398abe4437b889acb0a1c46191" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 40687 } }
I20260807 10:52:22.003948 20205 catalog_manager.cc:5495] T 2837431645b941db88584af86ccb707c P 2207ae398abe4437b889acb0a1c46191 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2207ae398abe4437b889acb0a1c46191 (127.3.73.195). New cstate: current_term: 1 leader_uuid: "2207ae398abe4437b889acb0a1c46191" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7665255ff3c481ca945de3da492cfcb" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 34227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79b69651e30f46a3858cbafce82e2824" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2207ae398abe4437b889acb0a1c46191" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 40687 } health_report { overall_health: HEALTHY } } }
I20260807 10:52:22.058724 21009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:52:22.050344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58624 (local address 127.3.73.195:40687)
0807 10:52:22.050909 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0807 10:52:22.050918 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:52:22.051488 (+ 570us) server_negotiation.cc:407] Connection header received
0807 10:52:22.051718 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:52:22.051722 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:52:22.051827 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:52:22.051974 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:52:22.052561 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:52:22.053574 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:52:22.055669 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:52:22.056328 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:52:22.056438 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:52:22.057171 (+ 733us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:52:22.057824 (+ 653us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:52:22.057958 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 10:52:22.058180 (+ 222us) server_negotiation.cc:299] Negotiation successful
0807 10:52:22.058328 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":237,"threads_started":1}
I20260807 10:52:22.068636 20450 raft_consensus.cc:1261] T 2837431645b941db88584af86ccb707c P a7665255ff3c481ca945de3da492cfcb [term 1 FOLLOWER]: Refusing update from remote peer 2207ae398abe4437b889acb0a1c46191: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:52:22.068660 20676 raft_consensus.cc:1261] T 2837431645b941db88584af86ccb707c P 79b69651e30f46a3858cbafce82e2824 [term 1 FOLLOWER]: Refusing update from remote peer 2207ae398abe4437b889acb0a1c46191: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:52:22.070009 20995 consensus_queue.cc:1035] T 2837431645b941db88584af86ccb707c P 2207ae398abe4437b889acb0a1c46191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a7665255ff3c481ca945de3da492cfcb" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 34227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:52:22.070456 20987 consensus_queue.cc:1035] T 2837431645b941db88584af86ccb707c P 2207ae398abe4437b889acb0a1c46191 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79b69651e30f46a3858cbafce82e2824" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:52:22.101294 20205 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:52:22.101824 20205 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:52:22.107196 20205 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7843cf9c69554d34872bb8cfe65fd06a: Sending DeleteTablet for 3 replicas of tablet 2837431645b941db88584af86ccb707c
I20260807 10:52:22.108846 20647 tablet_service.cc:1514] Processing DeleteTablet for tablet 2837431645b941db88584af86ccb707c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:52:22 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:37268
I20260807 10:52:22.108855 20422 tablet_service.cc:1514] Processing DeleteTablet for tablet 2837431645b941db88584af86ccb707c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:52:22 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:40356
I20260807 10:52:22.109076 20855 tablet_service.cc:1514] Processing DeleteTablet for tablet 2837431645b941db88584af86ccb707c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:52:22 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:58614
I20260807 10:52:22.114109 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 20324
I20260807 10:52:22.119786 21029 tablet_replica.cc:330] T 2837431645b941db88584af86ccb707c P 79b69651e30f46a3858cbafce82e2824: stopping tablet replica
I20260807 10:52:22.120563 21029 raft_consensus.cc:2229] T 2837431645b941db88584af86ccb707c P 79b69651e30f46a3858cbafce82e2824 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:52:22.121683 21029 raft_consensus.cc:2258] T 2837431645b941db88584af86ccb707c P 79b69651e30f46a3858cbafce82e2824 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:52:22.122968 21028 tablet_replica.cc:330] T 2837431645b941db88584af86ccb707c P a7665255ff3c481ca945de3da492cfcb: stopping tablet replica
I20260807 10:52:22.123605 21028 raft_consensus.cc:2229] T 2837431645b941db88584af86ccb707c P a7665255ff3c481ca945de3da492cfcb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:52:22.124682 21028 raft_consensus.cc:2258] T 2837431645b941db88584af86ccb707c P a7665255ff3c481ca945de3da492cfcb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:52:22.125897 21030 tablet_replica.cc:330] T 2837431645b941db88584af86ccb707c P 2207ae398abe4437b889acb0a1c46191: stopping tablet replica
I20260807 10:52:22.126475 21030 raft_consensus.cc:2229] T 2837431645b941db88584af86ccb707c P 2207ae398abe4437b889acb0a1c46191 [term 1 LEADER]: Raft consensus shutting down.
I20260807 10:52:22.127058 21028 ts_tablet_manager.cc:1905] T 2837431645b941db88584af86ccb707c P a7665255ff3c481ca945de3da492cfcb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:52:22.127450 21030 raft_consensus.cc:2258] T 2837431645b941db88584af86ccb707c P 2207ae398abe4437b889acb0a1c46191 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:52:22.129727 21030 ts_tablet_manager.cc:1905] T 2837431645b941db88584af86ccb707c P 2207ae398abe4437b889acb0a1c46191: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:52:22.134429 21029 ts_tablet_manager.cc:1905] T 2837431645b941db88584af86ccb707c P 79b69651e30f46a3858cbafce82e2824: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:52:22.136157 21028 ts_tablet_manager.cc:1918] T 2837431645b941db88584af86ccb707c P a7665255ff3c481ca945de3da492cfcb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:52:22.136473 21028 log.cc:1196] T 2837431645b941db88584af86ccb707c P a7665255ff3c481ca945de3da492cfcb: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-0/wal/wals/2837431645b941db88584af86ccb707c
I20260807 10:52:22.137231 21028 ts_tablet_manager.cc:1939] T 2837431645b941db88584af86ccb707c P a7665255ff3c481ca945de3da492cfcb: Deleting consensus metadata
I20260807 10:52:22.139796 20189 catalog_manager.cc:4841] TS a7665255ff3c481ca945de3da492cfcb (127.3.73.193:34227): tablet 2837431645b941db88584af86ccb707c (table test-table [id=31397f87b2b94e2dbda5b17c74417104]) successfully deleted
Aug 07 10:52:22 dist-test-slave-x5fp krb5kdc[19461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.193@KRBTEST.COM
I20260807 10:52:22.145213 21029 ts_tablet_manager.cc:1918] T 2837431645b941db88584af86ccb707c P 79b69651e30f46a3858cbafce82e2824: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:52:22.145576 21029 log.cc:1196] T 2837431645b941db88584af86ccb707c P 79b69651e30f46a3858cbafce82e2824: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-1/wal/wals/2837431645b941db88584af86ccb707c
I20260807 10:52:22.145673 21030 ts_tablet_manager.cc:1918] T 2837431645b941db88584af86ccb707c P 2207ae398abe4437b889acb0a1c46191: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:52:22.146047 21030 log.cc:1196] T 2837431645b941db88584af86ccb707c P 2207ae398abe4437b889acb0a1c46191: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786099767749326-3367-0/minicluster-data/ts-2/wal/wals/2837431645b941db88584af86ccb707c
I20260807 10:52:22.146405 21029 ts_tablet_manager.cc:1939] T 2837431645b941db88584af86ccb707c P 79b69651e30f46a3858cbafce82e2824: Deleting consensus metadata
I20260807 10:52:22.146770 21030 ts_tablet_manager.cc:1939] T 2837431645b941db88584af86ccb707c P 2207ae398abe4437b889acb0a1c46191: Deleting consensus metadata
I20260807 10:52:22.148944 20189 catalog_manager.cc:4841] TS 79b69651e30f46a3858cbafce82e2824 (127.3.73.194:43237): tablet 2837431645b941db88584af86ccb707c (table test-table [id=31397f87b2b94e2dbda5b17c74417104]) successfully deleted
I20260807 10:52:22.148981 20190 catalog_manager.cc:4841] TS 2207ae398abe4437b889acb0a1c46191 (127.3.73.195:40687): tablet 2837431645b941db88584af86ccb707c (table test-table [id=31397f87b2b94e2dbda5b17c74417104]) successfully deleted
I20260807 10:52:22.152943 20968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:52:22.117805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40360 (local address 127.3.73.193:34227)
0807 10:52:22.118084 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0807 10:52:22.118093 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:52:22.121512 (+ 3419us) server_negotiation.cc:407] Connection header received
0807 10:52:22.122323 (+ 811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:52:22.122330 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:52:22.122466 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:52:22.122564 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:52:22.127221 (+ 4657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:52:22.128582 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:52:22.131143 (+ 2561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:52:22.131980 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:52:22.132087 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:52:22.143140 (+ 11053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:52:22.143162 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:52:22.143183 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:52:22.143278 (+ 95us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:52:22.148893 (+ 5615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:52:22.150143 (+ 1250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:52:22.150155 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:52:22.150160 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:52:22.150359 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:52:22.150989 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:52:22.150993 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:52:22.150996 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:52:22.151470 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:52:22.151772 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0807 10:52:22.152166 (+ 394us) server_negotiation.cc:299] Negotiation successful
0807 10:52:22.152355 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260807 10:52:22.153928 20516 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:52:22.253813 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 20324
I20260807 10:52:22.283530 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 20527
Aug 07 10:52:22 dist-test-slave-x5fp krb5kdc[19461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.194@KRBTEST.COM
I20260807 10:52:22.301003 20972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:52:22.284410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37280 (local address 127.3.73.194:43237)
0807 10:52:22.284613 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 10:52:22.284619 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:52:22.285021 (+ 402us) server_negotiation.cc:407] Connection header received
0807 10:52:22.285255 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:52:22.285267 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:52:22.285370 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:52:22.285488 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:52:22.286953 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:52:22.287780 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:52:22.289259 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:52:22.289750 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:52:22.289907 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:52:22.294641 (+ 4734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:52:22.294657 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:52:22.294674 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:52:22.294719 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:52:22.297844 (+ 3125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:52:22.298591 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:52:22.298599 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:52:22.298602 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:52:22.298745 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:52:22.299179 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:52:22.299183 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:52:22.299186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:52:22.299590 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:52:22.299922 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0807 10:52:22.300252 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 10:52:22.300507 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 10:52:22.301718 20747 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:52:22.398056 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 20527
I20260807 10:52:22.433737 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 20760
Aug 07 10:52:22 dist-test-slave-x5fp krb5kdc[19461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.195@KRBTEST.COM
I20260807 10:52:22.455940 21009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:52:22.434683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58634 (local address 127.3.73.195:40687)
0807 10:52:22.434958 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0807 10:52:22.434966 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:52:22.435043 (+ 77us) server_negotiation.cc:407] Connection header received
0807 10:52:22.435257 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:52:22.435263 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:52:22.435372 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:52:22.435512 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:52:22.437541 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:52:22.438746 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:52:22.440860 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:52:22.441569 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:52:22.441712 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:52:22.447437 (+ 5725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:52:22.447453 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:52:22.447475 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:52:22.447529 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:52:22.452031 (+ 4502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:52:22.453081 (+ 1050us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:52:22.453091 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:52:22.453096 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:52:22.453288 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:52:22.453934 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:52:22.453939 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:52:22.453942 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:52:22.454481 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:52:22.454805 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0807 10:52:22.455163 (+ 358us) server_negotiation.cc:299] Negotiation successful
0807 10:52:22.455506 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260807 10:52:22.456914 20943 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:52:22.586247 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 20760
I20260807 10:52:22.614205 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 20166
I20260807 10:52:22.615566 20247 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:52:22.741330 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 20166
I20260807 10:52:22.775203 3367 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 10:52:23.122090 3367 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 10:52:23.122470 3367 mini_ranger.cc:66] Stopping Ranger...
I20260807 10:52:23.479369 3367 mini_ranger.cc:68] Stopped Ranger
2026-08-07 10:52:23.479 UTC [19507] LOG: received smart shutdown request
2026-08-07 10:52:23.484 UTC [19507] LOG: background worker "logical replication launcher" (PID 19521) exited with exit code 1
2026-08-07 10:52:23.484 UTC [19511] LOG: shutting down
2026-08-07 10:52:23.693 UTC [19507] LOG: database system is shut down
2026-08-07T10:52:23Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (60590 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:52:23 dist-test-slave-x5fp krb5kdc[21056](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:52:23 dist-test-slave-x5fp krb5kdc[21056](info): set up 2 sockets
Aug 07 10:52:23 dist-test-slave-x5fp krb5kdc[21056](info): commencing operation
krb5kdc: starting...
W20260807 10:52:25.343182 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.451s user 0.004s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:52:25 dist-test-slave-x5fp krb5kdc[21056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:52:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:52:25Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.3.73.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.3.73.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.3.73.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/rangeradmin_127.3.73.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.3.73.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/rangeradmin_127.3.73.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.3.73.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.3.73.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.3.73.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/rangerlookup_127.3.73.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.3.73.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/rangerlookup_127.3.73.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.3.73.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.3.73.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.3.73.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/HTTP_127.3.73.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.3.73.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/HTTP_127.3.73.212@KRBTEST.COM.keytab.
I20260807 10:52:25.825037 3367 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskRBZOut/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/postgres -l logfile start
2026-08-07 10:52:28.846 UTC [21102] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 10:52:28.846 UTC [21102] LOG: listening on IPv4 address "127.3.73.212", port 50801
2026-08-07 10:52:28.850 UTC [21102] LOG: listening on Unix socket "/tmp/.s.PGSQL.50801"
2026-08-07 10:52:28.877 UTC [21104] LOG: database system was shut down at 2026-08-07 10:52:27 UTC
2026-08-07 10:52:28.882 UTC [21102] LOG: database system is ready to accept connections
I20260807 10:52:30.115648 3367 mini_postgres.cc:96] Postgres bound to 50801
2026-08-07 10:52:30.143 UTC [21117] FATAL: database "slave" does not exist
127.3.73.212:50801 - accepting connections
I20260807 10:52:30.143836 3367 mini_ranger.cc:161] Starting Ranger...
I20260807 10:52:30.182294 3367 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 10:52:30.829612 3367 mini_ranger.cc:87] Created ranger Postgres database
I20260807 10:52:30.829732 3367 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ranger-admin
2026-08-07 10:52:31,167 [I] DB FLAVOR :POSTGRES
2026-08-07 10:52:31,167 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 10:52:31,167 [I] Checking connection..
2026-08-07 10:52:31,167 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:50801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 10:52:31,456 [I] Checking connection passed.
2026-08-07 10:52:31,456 [I] --------- Verifying version history table ---------
2026-08-07 10:52:31,457 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:50801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 10:52:31,746 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 10:52:31,746 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:50801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 10:52:32,028 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 10:52:32,029 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 10:52:32,029 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:50801/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 10:52:32.306 UTC [21218] WARNING: there is no transaction in progress
2026-08-07 10:52:32,317 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 10:52:32,317 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:50801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 10:52:32,594 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 10:52:32,595 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 10:52:32,595 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:50801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 10:52:32,873 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:50801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 10:52:33,152 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:50801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-x5fp.c.gcp-upstream.internal','N') ;"
2026-08-07 10:52:33,408 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:50801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 10:52:33,683 [I] Table x_portal_user does not exist in database ranger
2026-08-07 10:52:33,683 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:50801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 10:52:33,967 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 10:52:33,967 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:50801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 10:52:34,225 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:50801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 10:52:34,529 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 10:52:34,530 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:50801/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 10:52:35.657 UTC [21401] WARNING: there is no transaction in progress
2026-08-07 10:52:35.677 UTC [21401] WARNING: there is no transaction in progress
2026-08-07 10:52:35.696 UTC [21401] WARNING: there is no transaction in progress
2026-08-07 10:52:35.714 UTC [21401] WARNING: there is no transaction in progress
2026-08-07 10:52:35.733 UTC [21401] WARNING: there is no transaction in progress
2026-08-07 10:52:35.755 UTC [21401] WARNING: there is no transaction in progress
2026-08-07 10:52:35.824 UTC [21401] WARNING: there is no transaction in progress
2026-08-07 10:52:35.833 UTC [21401] WARNING: there is no transaction in progress
2026-08-07 10:52:35.849 UTC [21401] WARNING: there is no transaction in progress
2026-08-07 10:52:35.860 UTC [21401] WARNING: there is no transaction in progress
2026-08-07 10:52:35.872 UTC [21401] WARNING: there is no transaction in progress
2026-08-07 10:52:35.888 UTC [21401] WARNING: there is no transaction in progress
2026-08-07 10:52:35.901 UTC [21401] WARNING: there is no transaction in progress
2026-08-07 10:52:35.909 UTC [21401] WARNING: there is no transaction in progress
2026-08-07 10:52:35.918 UTC [21401] WARNING: there is no transaction in progress
2026-08-07 10:52:35.926 UTC [21401] WARNING: there is no transaction in progress
2026-08-07 10:52:36,746 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 10:52:36,747 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:50801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 10:52:37,025 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 10:52:37,025 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:50801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 10:52:37,330 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 10:52:37,330 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:50801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 10:52:37,612 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 10:52:37,613 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:50801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 10:52:37,906 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:50801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 10:52:38,274 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:50801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-x5fp.c.gcp-upstream.internal';"
2026-08-07 10:52:38,549 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 10:52:38,549 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:50801/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 10:52:38,833 [I] DB_PATCHES have already been applied
I20260807 10:52:38.840062 3367 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ranger-admin:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskRBZOut/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 10:52:38.840174 3367 mini_ranger.cc:193] Using host: 127.3.73.212
I20260807 10:52:38.865013 3367 mini_ranger.cc:227] Ranger admin URL: http://127.3.73.212:33973
Aug 07, 2026 10:52:39 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 10:52:39 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 10:52:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 10:52:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 10:52:39 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 10:52:40 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 10:52:40 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 10:52:40.377717 3367 mini_ranger.cc:160] Time spent starting Ranger: real 10.234s user 0.000s sys 0.076s
Aug 07, 2026 10:53:03 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 10:53:03 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 10:53:03 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 10:53:03 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 10:53:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 10:53:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 10:53:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 10:53:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 10:53:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 10:53:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 10:53:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 10:53:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 10:53:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 10:53:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 10:53:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 10:53:03 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 10:53:03 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 10:53:04 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 10:53:05.047760 3367 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.3.73.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.3.73.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.3.73.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/rangerkms_127.3.73.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.3.73.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/rangerkms_127.3.73.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.3.73.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.3.73.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.3.73.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/HTTP_127.3.73.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.3.73.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/HTTP_127.3.73.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 10:53:05 dist-test-slave-x5fp krb5kdc[21056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099985, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 10:53:05.309428 3367 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 10:53:05.343297 3367 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 10:53:06.490610 3367 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 10:53:06.490724 3367 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ranger-kms
I20260807 10:53:06.490757 3367 mini_ranger_kms.cc:227] Using postgres at 127.3.73.212:50801
2026-08-07 10:53:06,889 [I] DB FLAVOR :POSTGRES
2026-08-07 10:53:06,889 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 10:53:06,889 [I] Checking connection
2026-08-07 10:53:06,890 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:50801/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 10:53:07,192 [I] connection success
2026-08-07 10:53:07,192 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 10:53:07,192 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 10:53:07,192 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:50801/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 10:53:07,461 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 10:53:07,462 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 10:53:07,462 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 10:53:07,462 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.73.212:50801/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 10:53:07,781 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 10:53:08.353533 3367 mini_ranger_kms.cc:310] Created kms service
I20260807 10:53:08.564759 3367 mini_ranger_kms.cc:326] Created kudu user
I20260807 10:53:08.617398 3367 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 10:53:08 dist-test-slave-x5fp krb5kdc[21056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099985, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.3.73.212@KRBTEST.COM
I20260807 10:53:09.117190 3367 mini_ranger_kms.cc:384] Added ranger policy
I20260807 10:53:09.117588 3367 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ranger-kms:/tmp/dist-test-taskRBZOut/build/asan/bin/postgresql.jar:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskRBZOut/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 10:53:09.117658 3367 mini_ranger_kms.cc:240] Using host: 127.3.73.212
I20260807 10:53:09.146636 3367 mini_ranger_kms.cc:276] Ranger KMS PID: 21677
I20260807 10:53:09.146811 3367 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.3.73.212:43333
Aug 07, 2026 10:53:10 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 10:53:10 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 10:53:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 10:53:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 10:53:10 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 10:53:10 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 10:53:11 dist-test-slave-x5fp krb5kdc[21056](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786099991, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.73.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:53:11.974504 3367 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.665s user 0.066s sys 0.379s
I20260807 10:53:11.974838 3367 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 10:53:11.975034 3367 mini_ranger_kms.cc:401] 127.3.73.212:43333/kms/v1/keys
Aug 07 10:53:12 dist-test-slave-x5fp krb5kdc[21056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099991, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.73.212@KRBTEST.COM for HTTP/127.3.73.212@KRBTEST.COM
Aug 07 10:53:12 dist-test-slave-x5fp krb5kdc[21056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099991, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.73.212@KRBTEST.COM for HTTP/127.3.73.212@KRBTEST.COM
Aug 07 10:53:15 dist-test-slave-x5fp krb5kdc[21056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.3.73.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
I20260807 10:53:15.345638 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:42827
--webserver_interface=127.3.73.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:39741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:42827
--ranger_config_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.73.212:43333/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:53:15.715190 21759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:53:15.715549 21759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:53:15.715708 21759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:53:15.723273 21759 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:53:15.723412 21759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:53:15.723495 21759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:53:15.723600 21759 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:53:15.723680 21759 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:53:15.734361 21759 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:39741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/master-0/wal
--ranger_kms_url=127.3.73.212:43333/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:42827
--ranger_config_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:42827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:53:15.735821 21759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:53:15.737393 21759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:53:15.751701 21765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:15.751678 21766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:15.751654 21768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:53:15.752477 21759 server_base.cc:1034] running on GCE node
I20260807 10:53:15.753463 21759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:53:15.755000 21759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:53:15.756250 21759 hybrid_clock.cc:648] HybridClock initialized: now 1786099995756170 us; error 51 us; skew 500 ppm
I20260807 10:53:15.756647 21759 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:53:15 dist-test-slave-x5fp krb5kdc[21056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099995, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:53:15.760037 21759 init.cc:377] Logged in from keytab as kudu/127.3.73.254@KRBTEST.COM (short username kudu)
I20260807 10:53:15.761623 21759 webserver.cc:460] Webserver started at http://127.3.73.254:37667/ using document root <none> and password file <none>
I20260807 10:53:15.762311 21759 fs_manager.cc:359] Metadata directory not provided
I20260807 10:53:15.762431 21759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:53:15.762713 21759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 10:53:15 dist-test-slave-x5fp krb5kdc[21056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099995, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for HTTP/127.3.73.212@KRBTEST.COM
I20260807 10:53:15.814431 21759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "aa18b7264c0347678a0cc0a30c4fdf03"
format_stamp: "Formatted at 2026-08-07 10:53:15 on dist-test-slave-x5fp"
server_key: "85717bde78eebe14e6c4d9d1738c1847"
server_key_iv: "fd1fdc1628b24ed432acf97a91a316e1"
server_key_version: "kuduclusterkey@0"
I20260807 10:53:15.815410 21759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "aa18b7264c0347678a0cc0a30c4fdf03"
format_stamp: "Formatted at 2026-08-07 10:53:15 on dist-test-slave-x5fp"
server_key: "85717bde78eebe14e6c4d9d1738c1847"
server_key_iv: "fd1fdc1628b24ed432acf97a91a316e1"
server_key_version: "kuduclusterkey@0"
I20260807 10:53:15.821552 21759 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260807 10:53:15.841601 21779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:15.843429 21759 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 10:53:15.843658 21759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "aa18b7264c0347678a0cc0a30c4fdf03"
format_stamp: "Formatted at 2026-08-07 10:53:15 on dist-test-slave-x5fp"
server_key: "85717bde78eebe14e6c4d9d1738c1847"
server_key_iv: "fd1fdc1628b24ed432acf97a91a316e1"
server_key_version: "kuduclusterkey@0"
I20260807 10:53:15.843945 21759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:53:15.876549 21759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:53:15.881753 21759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:53:15.882213 21759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:53:15.903389 21759 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:42827
I20260807 10:53:15.903446 21861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:42827 every 8 connection(s)
I20260807 10:53:15.904907 21759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:53:15.909296 21862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:53:15.912310 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 21759
I20260807 10:53:15.912554 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:53:15.913488 3367 external_mini_cluster.cc:1428] Setting key af5b51f452c4943ecceef3fb59a6326d
I20260807 10:53:15.923818 21862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03: Bootstrap starting.
I20260807 10:53:15.928364 21862 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03: Neither blocks nor log segments found. Creating new log.
I20260807 10:53:15.929924 21862 log.cc:824] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03: Log is configured to *not* fsync() on all Append() calls
Aug 07 10:53:15 dist-test-slave-x5fp krb5kdc[21056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:53:15.933805 21862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03: No bootstrap required, opened a new log
I20260807 10:53:15.941185 21867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:15.915997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60736 (local address 127.3.73.254:42827)
0807 10:53:15.916952 (+ 955us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:15.916965 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:15.917015 (+ 50us) server_negotiation.cc:407] Connection header received
0807 10:53:15.918790 (+ 1775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:15.918816 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:15.919271 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:15.919651 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:15.921696 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:15.923130 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:15.925135 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:15.925913 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:15.926072 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:15.931841 (+ 5769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:15.931851 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:15.931870 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:15.931911 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:15.935989 (+ 4078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:15.936847 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:15.936856 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:15.936866 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:15.937096 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:15.937518 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:15.937523 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:15.937527 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:15.938341 (+ 814us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:15.938684 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:15.939061 (+ 377us) server_negotiation.cc:299] Negotiation successful
0807 10:53:15.939573 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":719,"thread_start_us":454,"threads_started":1}
I20260807 10:53:15.944414 21862 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa18b7264c0347678a0cc0a30c4fdf03" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 42827 } }
I20260807 10:53:15.945003 21862 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:53:15.945134 21862 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa18b7264c0347678a0cc0a30c4fdf03, State: Initialized, Role: FOLLOWER
I20260807 10:53:15.945747 21862 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa18b7264c0347678a0cc0a30c4fdf03" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 42827 } }
I20260807 10:53:15.946003 21862 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:53:15.946151 21862 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:53:15.946328 21862 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:53:15.948889 21862 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa18b7264c0347678a0cc0a30c4fdf03" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 42827 } }
I20260807 10:53:15.949522 21862 leader_election.cc:302] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa18b7264c0347678a0cc0a30c4fdf03; no voters:
I20260807 10:53:15.950280 21862 leader_election.cc:288] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:53:15.950412 21870 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:53:15.951941 21870 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03 [term 1 LEADER]: Becoming Leader. State: Replica: aa18b7264c0347678a0cc0a30c4fdf03, State: Running, Role: LEADER
I20260807 10:53:15.952589 21870 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa18b7264c0347678a0cc0a30c4fdf03" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 42827 } }
I20260807 10:53:15.953389 21862 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:53:15.955892 21873 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa18b7264c0347678a0cc0a30c4fdf03. Latest consensus state: current_term: 1 leader_uuid: "aa18b7264c0347678a0cc0a30c4fdf03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa18b7264c0347678a0cc0a30c4fdf03" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 42827 } } }
I20260807 10:53:15.956445 21873 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03 [sys.catalog]: This master's current role is: LEADER
I20260807 10:53:15.957314 21872 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa18b7264c0347678a0cc0a30c4fdf03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa18b7264c0347678a0cc0a30c4fdf03" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 42827 } } }
I20260807 10:53:15.957815 21872 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03 [sys.catalog]: This master's current role is: LEADER
I20260807 10:53:15.958261 21881 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:53:15.971776 21881 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:53:15.973256 21862 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 10:53:15.993902 21881 catalog_manager.cc:1347] Generated new cluster ID: ef07b1f345c346fd891b9aaec05bd787
I20260807 10:53:15.994079 21881 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:53:16.032346 21881 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:53:16.034699 21881 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:53:16.050699 21881 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03: Generated new TSK 0
I20260807 10:53:16.051960 21881 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.73.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab.
Entry for principal kudu/127.3.73.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab.
I20260807 10:53:16.887137 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.73.212:43333/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.193:0
--local_ip_for_outbound_sockets=127.3.73.193
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:42827
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:39741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 10:53:17 dist-test-slave-x5fp krb5kdc[21056](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786099997, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 10:53:17.452507 21938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:53:17.452996 21938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:53:17.453166 21938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:53:17.466416 21938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:53:17.466621 21938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:53:17.466849 21938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.193
I20260807 10:53:17.484809 21938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:39741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.3.73.212:43333/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:42827
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.193
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:53:17.486855 21938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:53:17.488977 21938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:53:17.513688 21954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:17.515121 21955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:17.517158 21957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:53:17.517174 21938 server_base.cc:1034] running on GCE node
I20260807 10:53:17.518163 21938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:53:17.519459 21938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:53:17.520825 21938 hybrid_clock.cc:648] HybridClock initialized: now 1786099997520713 us; error 38 us; skew 500 ppm
I20260807 10:53:17.521271 21938 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:53:17 dist-test-slave-x5fp krb5kdc[21056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099997, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:53:17.525712 21938 init.cc:377] Logged in from keytab as kudu/127.3.73.193@KRBTEST.COM (short username kudu)
I20260807 10:53:17.528110 21938 webserver.cc:460] Webserver started at http://127.3.73.193:46875/ using document root <none> and password file <none>
I20260807 10:53:17.529299 21938 fs_manager.cc:359] Metadata directory not provided
I20260807 10:53:17.529491 21938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:53:17.529922 21938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 10:53:17 dist-test-slave-x5fp krb5kdc[21056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099997, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for HTTP/127.3.73.212@KRBTEST.COM
I20260807 10:53:17.561244 21938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-0/data/instance:
uuid: "c96cd230adcd495e8d31a810c75f26a4"
format_stamp: "Formatted at 2026-08-07 10:53:17 on dist-test-slave-x5fp"
server_key: "e2fddb4615fd6f54e755a5436eff3c38"
server_key_iv: "9064bea98f0a4e58944fd0016a46550d"
server_key_version: "kuduclusterkey@0"
I20260807 10:53:17.562899 21938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance:
uuid: "c96cd230adcd495e8d31a810c75f26a4"
format_stamp: "Formatted at 2026-08-07 10:53:17 on dist-test-slave-x5fp"
server_key: "e2fddb4615fd6f54e755a5436eff3c38"
server_key_iv: "9064bea98f0a4e58944fd0016a46550d"
server_key_version: "kuduclusterkey@0"
I20260807 10:53:17.579217 21938 fs_manager.cc:691] Time spent creating directory manager: real 0.016s user 0.008s sys 0.000s
I20260807 10:53:17.612581 21969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:17.614997 21938 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.006s sys 0.001s
I20260807 10:53:17.615239 21938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-0/wal
uuid: "c96cd230adcd495e8d31a810c75f26a4"
format_stamp: "Formatted at 2026-08-07 10:53:17 on dist-test-slave-x5fp"
server_key: "e2fddb4615fd6f54e755a5436eff3c38"
server_key_iv: "9064bea98f0a4e58944fd0016a46550d"
server_key_version: "kuduclusterkey@0"
I20260807 10:53:17.615537 21938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:53:17.648254 21938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:53:17.654006 21938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:53:17.654376 21938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:53:17.661875 21938 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:53:17.665501 21938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:53:17.665627 21938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:17.665761 21938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:53:17.665817 21938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 10:53:17 dist-test-slave-x5fp krb5kdc[21056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099997, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for HTTP/127.3.73.212@KRBTEST.COM
I20260807 10:53:17.853174 21938 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.193:42013
I20260807 10:53:17.853195 22139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.193:42013 every 8 connection(s)
I20260807 10:53:17.854912 21938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
I20260807 10:53:17.865216 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 21938
I20260807 10:53:17.865540 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance
I20260807 10:53:17.866436 3367 external_mini_cluster.cc:1428] Setting key c8d7f16c3fd7457ecd7f8f6944d51612
Aug 07 10:53:17 dist-test-slave-x5fp krb5kdc[21056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099997, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:53:17.890969 22144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:17.859780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:42827 (local address 127.3.73.193:40061)
0807 10:53:17.862060 (+ 2280us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:17.862136 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0807 10:53:17.864824 (+ 2688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:53:17.866672 (+ 1848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:53:17.866688 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:53:17.867503 (+ 815us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:53:17.869487 (+ 1984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:17.869522 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:17.871350 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:17.871358 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:17.873889 (+ 2531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:17.873915 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:17.875457 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:17.875466 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:17.875842 (+ 376us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:17.876849 (+ 1007us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:53:17.876887 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:53:17.880141 (+ 3254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:53:17.885157 (+ 5016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:17.885170 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:17.885195 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:17.885969 (+ 774us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:17.886537 (+ 568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:17.886543 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:17.886548 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:17.886908 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:17.887778 (+ 870us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:53:17.887788 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:53:17.888318 (+ 530us) client_negotiation.cc:769] Sending connection context
0807 10:53:17.888691 (+ 373us) client_negotiation.cc:240] Negotiation successful
0807 10:53:17.889375 (+ 684us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1902,"thread_start_us":6397,"threads_started":1}
I20260807 10:53:17.892365 22143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:17.859016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:40061 (local address 127.3.73.254:42827)
0807 10:53:17.865791 (+ 6775us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:17.865798 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:17.865821 (+ 23us) server_negotiation.cc:407] Connection header received
0807 10:53:17.865895 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:17.865900 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:17.866009 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:17.866141 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:17.869758 (+ 3617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:17.871091 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:17.874161 (+ 3070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:17.874914 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:17.875062 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:17.880514 (+ 5452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:17.880523 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:17.880529 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:17.880596 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:17.884867 (+ 4271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:17.886166 (+ 1299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:17.886174 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:17.886178 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:17.886373 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:17.887097 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:17.887106 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:17.887112 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:17.887598 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:17.887785 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:17.891676 (+ 3891us) server_negotiation.cc:299] Negotiation successful
0807 10:53:17.891964 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6627,"thread_start_us":6555,"threads_started":1}
I20260807 10:53:17.894340 22140 heartbeater.cc:347] Connected to a master server at 127.3.73.254:42827
I20260807 10:53:17.894711 22140 heartbeater.cc:464] Registering TS with master...
I20260807 10:53:17.895651 22140 heartbeater.cc:511] Master 127.3.73.254:42827 requested a full tablet report, sending...
I20260807 10:53:17.898713 21803 ts_manager.cc:194] Registered new tserver with Master: c96cd230adcd495e8d31a810c75f26a4 (127.3.73.193:42013)
I20260807 10:53:17.901129 21803 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.193@KRBTEST.COM'} at 127.3.73.193:40061
WARNING: no policy specified for kudu/127.3.73.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab.
Entry for principal kudu/127.3.73.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab.
I20260807 10:53:17.969599 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.73.212:43333/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.194:0
--local_ip_for_outbound_sockets=127.3.73.194
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:42827
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:39741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 10:53:18 dist-test-slave-x5fp krb5kdc[21056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099997, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for HTTP/127.3.73.212@KRBTEST.COM
W20260807 10:53:18.520056 22151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:53:18.520448 22151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:53:18.520610 22151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:53:18.528474 22151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:53:18.528668 22151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:53:18.528875 22151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.194
I20260807 10:53:18.539391 22151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:39741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.3.73.212:43333/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:42827
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.194
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:53:18.540915 22151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:53:18.542265 22151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:53:18.557142 22171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:18.557178 22170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:18.557235 22176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:53:18.558265 22151 server_base.cc:1034] running on GCE node
I20260807 10:53:18.559029 22151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:53:18.560225 22151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:53:18.561496 22151 hybrid_clock.cc:648] HybridClock initialized: now 1786099998561433 us; error 57 us; skew 500 ppm
I20260807 10:53:18.561939 22151 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:53:18 dist-test-slave-x5fp krb5kdc[21056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099998, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:53:18.565871 22151 init.cc:377] Logged in from keytab as kudu/127.3.73.194@KRBTEST.COM (short username kudu)
I20260807 10:53:18.567754 22151 webserver.cc:460] Webserver started at http://127.3.73.194:44129/ using document root <none> and password file <none>
I20260807 10:53:18.568554 22151 fs_manager.cc:359] Metadata directory not provided
I20260807 10:53:18.568787 22151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:53:18.569147 22151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 10:53:18 dist-test-slave-x5fp krb5kdc[21056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099998, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for HTTP/127.3.73.212@KRBTEST.COM
I20260807 10:53:18.587842 22151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-1/data/instance:
uuid: "d81c4bbc01254e8eaee0c19e0d3d775f"
format_stamp: "Formatted at 2026-08-07 10:53:18 on dist-test-slave-x5fp"
server_key: "46592f31c22d635ad5b83e6ef08f1e7b"
server_key_iv: "217437aec59884c4a4eb4532b76ec4d5"
server_key_version: "kuduclusterkey@0"
I20260807 10:53:18.589061 22151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance:
uuid: "d81c4bbc01254e8eaee0c19e0d3d775f"
format_stamp: "Formatted at 2026-08-07 10:53:18 on dist-test-slave-x5fp"
server_key: "46592f31c22d635ad5b83e6ef08f1e7b"
server_key_iv: "217437aec59884c4a4eb4532b76ec4d5"
server_key_version: "kuduclusterkey@0"
I20260807 10:53:18.595577 22151 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 10:53:18.610961 22185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:18.612668 22151 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 10:53:18.612924 22151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-1/wal
uuid: "d81c4bbc01254e8eaee0c19e0d3d775f"
format_stamp: "Formatted at 2026-08-07 10:53:18 on dist-test-slave-x5fp"
server_key: "46592f31c22d635ad5b83e6ef08f1e7b"
server_key_iv: "217437aec59884c4a4eb4532b76ec4d5"
server_key_version: "kuduclusterkey@0"
I20260807 10:53:18.613186 22151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:53:18.641449 22151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:53:18.646587 22151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:53:18.646950 22151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:53:18.648252 22151 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:53:18.650024 22151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:53:18.650134 22151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:18.650285 22151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:53:18.650354 22151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:18.689611 22151 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.194:34623
I20260807 10:53:18.689630 22381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.194:34623 every 8 connection(s)
I20260807 10:53:18.691205 22151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
I20260807 10:53:18.698468 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 22151
I20260807 10:53:18.698663 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance
I20260807 10:53:18.699247 3367 external_mini_cluster.cc:1428] Setting key 6c73051be8074970ff921444daa53451
Aug 07 10:53:18 dist-test-slave-x5fp krb5kdc[21056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099998, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:53:18.720785 22384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:18.694362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:38093 (local address 127.3.73.254:42827)
0807 10:53:18.694889 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:18.694896 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:18.696171 (+ 1275us) server_negotiation.cc:407] Connection header received
0807 10:53:18.698415 (+ 2244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:18.698421 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:18.698540 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:18.698694 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:18.700601 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:18.701770 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:18.705105 (+ 3335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:18.705772 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:18.705921 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:18.711834 (+ 5913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:18.711843 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:18.711849 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:18.711898 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:18.716056 (+ 4158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:18.717535 (+ 1479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:18.717542 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:18.717546 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:18.717852 (+ 306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:18.718566 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:18.718571 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:18.718575 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:18.719002 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:18.719168 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:18.720050 (+ 882us) server_negotiation.cc:299] Negotiation successful
0807 10:53:18.720343 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":255,"threads_started":1}
I20260807 10:53:18.722057 22387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:18.694979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:42827 (local address 127.3.73.194:38093)
0807 10:53:18.695960 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:18.696012 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 10:53:18.698083 (+ 2071us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:53:18.699004 (+ 921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:53:18.699014 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:53:18.699516 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:53:18.700374 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:18.700403 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:18.702001 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:18.702007 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:18.704546 (+ 2539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:18.704582 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:18.706081 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:18.706092 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:18.706478 (+ 386us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:18.707440 (+ 962us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:53:18.707482 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:53:18.711457 (+ 3975us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:53:18.716309 (+ 4852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:18.716324 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:18.716352 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:18.717343 (+ 991us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:18.718005 (+ 662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:18.718013 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:18.718020 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:18.718414 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:18.719190 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:53:18.719197 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:53:18.719579 (+ 382us) client_negotiation.cc:769] Sending connection context
0807 10:53:18.719937 (+ 358us) client_negotiation.cc:240] Negotiation successful
0807 10:53:18.720528 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":685,"thread_start_us":435,"threads_started":1}
I20260807 10:53:18.723578 22382 heartbeater.cc:347] Connected to a master server at 127.3.73.254:42827
I20260807 10:53:18.723996 22382 heartbeater.cc:464] Registering TS with master...
I20260807 10:53:18.724879 22382 heartbeater.cc:511] Master 127.3.73.254:42827 requested a full tablet report, sending...
I20260807 10:53:18.726614 21803 ts_manager.cc:194] Registered new tserver with Master: d81c4bbc01254e8eaee0c19e0d3d775f (127.3.73.194:34623)
I20260807 10:53:18.727466 21803 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.194@KRBTEST.COM'} at 127.3.73.194:38093
WARNING: no policy specified for kudu/127.3.73.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab.
Entry for principal kudu/127.3.73.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab.
I20260807 10:53:18.797778 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.73.212:43333/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.195:0
--local_ip_for_outbound_sockets=127.3.73.195
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:42827
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:39741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 10:53:18.905938 22140 heartbeater.cc:503] Master 127.3.73.254:42827 was elected leader, sending a full tablet report...
W20260807 10:53:19.191766 22391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:53:19.192162 22391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:53:19.192309 22391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:53:19.199731 22391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:53:19.199877 22391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:53:19.199997 22391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.195
I20260807 10:53:19.210386 22391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:39741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.3.73.212:43333/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:42827
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.195
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:53:19.211802 22391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:53:19.213425 22391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:53:19.229158 22398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:19.229094 22401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:19.229184 22399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:53:19.230269 22391 server_base.cc:1034] running on GCE node
I20260807 10:53:19.231029 22391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:53:19.232132 22391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:53:19.233373 22391 hybrid_clock.cc:648] HybridClock initialized: now 1786099999233325 us; error 47 us; skew 500 ppm
I20260807 10:53:19.233750 22391 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:53:19 dist-test-slave-x5fp krb5kdc[21056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099999, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:53:19.242048 22391 init.cc:377] Logged in from keytab as kudu/127.3.73.195@KRBTEST.COM (short username kudu)
I20260807 10:53:19.247212 22391 webserver.cc:460] Webserver started at http://127.3.73.195:37239/ using document root <none> and password file <none>
I20260807 10:53:19.248143 22391 fs_manager.cc:359] Metadata directory not provided
I20260807 10:53:19.248327 22391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:53:19.248687 22391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 10:53:19 dist-test-slave-x5fp krb5kdc[21056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099999, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for HTTP/127.3.73.212@KRBTEST.COM
I20260807 10:53:19.264046 22391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-2/data/instance:
uuid: "4209b5f7d4d34c82bb07dcb0b78463d3"
format_stamp: "Formatted at 2026-08-07 10:53:19 on dist-test-slave-x5fp"
server_key: "ecea9bd167d6ed4c44a67767427aa1b6"
server_key_iv: "8d0be2262a6fe01e186cc556213bf701"
server_key_version: "kuduclusterkey@0"
I20260807 10:53:19.265174 22391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance:
uuid: "4209b5f7d4d34c82bb07dcb0b78463d3"
format_stamp: "Formatted at 2026-08-07 10:53:19 on dist-test-slave-x5fp"
server_key: "ecea9bd167d6ed4c44a67767427aa1b6"
server_key_iv: "8d0be2262a6fe01e186cc556213bf701"
server_key_version: "kuduclusterkey@0"
I20260807 10:53:19.271648 22391 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 10:53:19.289326 22414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:19.291440 22391 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 10:53:19.291661 22391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-2/wal
uuid: "4209b5f7d4d34c82bb07dcb0b78463d3"
format_stamp: "Formatted at 2026-08-07 10:53:19 on dist-test-slave-x5fp"
server_key: "ecea9bd167d6ed4c44a67767427aa1b6"
server_key_iv: "8d0be2262a6fe01e186cc556213bf701"
server_key_version: "kuduclusterkey@0"
I20260807 10:53:19.291950 22391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:53:19.312183 22391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:53:19.316627 22391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:53:19.316999 22391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:53:19.318204 22391 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:53:19.319739 22391 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:53:19.319814 22391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:19.319980 22391 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:53:19.320041 22391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:19.360636 22391 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.195:43601
I20260807 10:53:19.360679 22578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.195:43601 every 8 connection(s)
I20260807 10:53:19.362188 22391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
I20260807 10:53:19.368629 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 22391
I20260807 10:53:19.368952 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance
I20260807 10:53:19.369508 3367 external_mini_cluster.cc:1428] Setting key c6c0b1fb4dfcc7666e8c5d4d68508b9c
Aug 07 10:53:19 dist-test-slave-x5fp krb5kdc[21056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099999, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:53:19.385767 22583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:19.365137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:54421 (local address 127.3.73.254:42827)
0807 10:53:19.365712 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:19.365717 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:19.367192 (+ 1475us) server_negotiation.cc:407] Connection header received
0807 10:53:19.369341 (+ 2149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:19.369348 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:19.369464 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:19.369595 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:19.371648 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:19.372565 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:19.374474 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:19.374914 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:19.375060 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:19.378964 (+ 3904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:19.378975 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:19.378979 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:19.379016 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:19.381995 (+ 2979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:19.382946 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:19.382951 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:19.382954 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:19.383102 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:19.383599 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:19.383603 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:19.383606 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:19.383914 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:19.384089 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:19.385094 (+ 1005us) server_negotiation.cc:299] Negotiation successful
0807 10:53:19.385368 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":291,"threads_started":1}
I20260807 10:53:19.387013 22585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:19.365835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:42827 (local address 127.3.73.195:54421)
0807 10:53:19.366921 (+ 1086us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:19.366983 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 10:53:19.368979 (+ 1996us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:53:19.369945 (+ 966us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:53:19.369960 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:53:19.370587 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:53:19.371460 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:19.371482 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:19.372762 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:19.372767 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:19.374303 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:19.374318 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:19.375181 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:19.375188 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:19.375467 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:19.376317 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:53:19.376344 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:53:19.378671 (+ 2327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:53:19.382216 (+ 3545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:19.382225 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:19.382249 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:19.382784 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:19.383243 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:19.383247 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:19.383249 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:19.383458 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:19.384063 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:53:19.384073 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:53:19.384542 (+ 469us) client_negotiation.cc:769] Sending connection context
0807 10:53:19.384955 (+ 413us) client_negotiation.cc:240] Negotiation successful
0807 10:53:19.385573 (+ 618us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":774,"thread_start_us":492,"threads_started":1}
I20260807 10:53:19.388535 22579 heartbeater.cc:347] Connected to a master server at 127.3.73.254:42827
I20260807 10:53:19.388943 22579 heartbeater.cc:464] Registering TS with master...
I20260807 10:53:19.389937 22579 heartbeater.cc:511] Master 127.3.73.254:42827 requested a full tablet report, sending...
I20260807 10:53:19.391808 21803 ts_manager.cc:194] Registered new tserver with Master: 4209b5f7d4d34c82bb07dcb0b78463d3 (127.3.73.195:43601)
I20260807 10:53:19.393049 21803 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.195@KRBTEST.COM'} at 127.3.73.195:54421
I20260807 10:53:19.400031 3367 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 10:53:19.402004 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 21938
Aug 07 10:53:19 dist-test-slave-x5fp krb5kdc[21056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.193@KRBTEST.COM
I20260807 10:53:19.420811 22587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:19.403419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41906 (local address 127.3.73.193:42013)
0807 10:53:19.404141 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:19.404149 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:19.404177 (+ 28us) server_negotiation.cc:407] Connection header received
0807 10:53:19.404238 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:19.404244 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:19.404490 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:19.404630 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:19.406400 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:19.407457 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:19.409162 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:19.409799 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:19.409917 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:19.414671 (+ 4754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:19.414687 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:19.414704 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:19.414752 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:19.417816 (+ 3064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:19.418690 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:19.418698 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:19.418701 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:19.418903 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:19.419256 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:19.419260 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:19.419261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:19.419621 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:19.419891 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:19.420122 (+ 231us) server_negotiation.cc:299] Negotiation successful
0807 10:53:19.420340 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":409,"threads_started":1}
I20260807 10:53:19.422441 22134 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:53:19.513052 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 21938
I20260807 10:53:19.539113 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 22151
Aug 07 10:53:19 dist-test-slave-x5fp krb5kdc[21056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.194@KRBTEST.COM
I20260807 10:53:19.568387 22590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:19.540390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47332 (local address 127.3.73.194:34623)
0807 10:53:19.541085 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:19.541093 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:19.541122 (+ 29us) server_negotiation.cc:407] Connection header received
0807 10:53:19.541201 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:19.541209 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:19.541430 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:19.541576 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:19.543091 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:19.544476 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:19.546292 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:19.547019 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:19.547156 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:19.560612 (+ 13456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:19.560633 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:19.560651 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:19.560698 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:19.564148 (+ 3450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:19.565606 (+ 1458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:19.565619 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:19.565622 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:19.565811 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:19.566341 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:19.566345 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:19.566347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:19.566746 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:19.567028 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:19.567711 (+ 683us) server_negotiation.cc:299] Negotiation successful
0807 10:53:19.568055 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":389,"threads_started":1}
I20260807 10:53:19.570919 22375 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:53:19.658365 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 22151
I20260807 10:53:19.685886 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 22391
Aug 07 10:53:19 dist-test-slave-x5fp krb5kdc[21056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786099995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.195@KRBTEST.COM
I20260807 10:53:19.706290 22592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:19.687049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50832 (local address 127.3.73.195:43601)
0807 10:53:19.687848 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:19.687856 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:19.687891 (+ 35us) server_negotiation.cc:407] Connection header received
0807 10:53:19.687956 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:19.687964 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:19.688199 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:19.688341 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:19.689937 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:19.691145 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:19.692782 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:19.693450 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:19.693580 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:19.699029 (+ 5449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:19.699044 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:19.699061 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:19.699106 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:19.702525 (+ 3419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:19.703671 (+ 1146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:19.703679 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:19.703681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:19.703890 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:19.704425 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:19.704430 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:19.704432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:19.705010 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:19.705322 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:19.705652 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 10:53:19.705958 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":376,"threads_started":1}
I20260807 10:53:19.707922 22573 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:53:19.801117 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 22391
I20260807 10:53:19.827612 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 21759
I20260807 10:53:19.829039 21857 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:53:19.941771 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 21759
I20260807 10:53:19.980065 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:42827
--webserver_interface=127.3.73.254
--webserver_port=37667
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:39741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:42827
--ranger_config_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.73.212:43333/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.3.73.212:43333/kms with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:53:20.370882 22598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:53:20.371250 22598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:53:20.371353 22598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:53:20.379552 22598 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:53:20.379729 22598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:53:20.379779 22598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:53:20.379839 22598 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:53:20.379891 22598 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:53:20.391366 22598 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:39741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.3.73.212:43333/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:42827
--ranger_config_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:42827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=37667
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:53:20.392938 22598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:53:20.394531 22598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:53:20.410251 22610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:20.410403 22608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:20.410454 22604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:53:20.411413 22598 server_base.cc:1034] running on GCE node
I20260807 10:53:20.412595 22598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:53:20.414373 22598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:53:20.415663 22598 hybrid_clock.cc:648] HybridClock initialized: now 1786100000415630 us; error 88 us; skew 500 ppm
I20260807 10:53:20.416087 22598 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:53:20 dist-test-slave-x5fp krb5kdc[21056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100000, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:53:20.419842 22598 init.cc:377] Logged in from keytab as kudu/127.3.73.254@KRBTEST.COM (short username kudu)
I20260807 10:53:20.421691 22598 webserver.cc:460] Webserver started at http://127.3.73.254:37667/ using document root <none> and password file <none>
I20260807 10:53:20.422652 22598 fs_manager.cc:359] Metadata directory not provided
I20260807 10:53:20.422850 22598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 10:53:20 dist-test-slave-x5fp krb5kdc[21056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100000, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for HTTP/127.3.73.212@KRBTEST.COM
I20260807 10:53:20.484890 22598 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 10:53:20.488312 22626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:20.489895 22598 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 10:53:20.490142 22598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "aa18b7264c0347678a0cc0a30c4fdf03"
format_stamp: "Formatted at 2026-08-07 10:53:15 on dist-test-slave-x5fp"
server_key: "85717bde78eebe14e6c4d9d1738c1847"
server_key_iv: "fd1fdc1628b24ed432acf97a91a316e1"
server_key_version: "kuduclusterkey@0"
I20260807 10:53:20.491242 22598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:53:20.507433 22598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:53:20.512530 22598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:53:20.512965 22598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:53:20.534272 22598 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:42827
I20260807 10:53:20.534314 22703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:42827 every 8 connection(s)
I20260807 10:53:20.535977 22598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:53:20.536810 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 22598
I20260807 10:53:20.537914 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.73.212:43333/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.193:42013
--local_ip_for_outbound_sockets=127.3.73.193
--tserver_master_addrs=127.3.73.254:42827
--webserver_port=46875
--webserver_interface=127.3.73.193
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:39741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.3.73.212:43333/kms with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 10:53:20.544907 22704 sys_catalog.cc:263] Verifying existing consensus state
I20260807 10:53:20.548583 22704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03: Bootstrap starting.
I20260807 10:53:20.568929 22704 log.cc:824] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03: Log is configured to *not* fsync() on all Append() calls
I20260807 10:53:20.579020 22704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 10:53:20.579674 22704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03: Bootstrap complete.
I20260807 10:53:20.589403 22704 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa18b7264c0347678a0cc0a30c4fdf03" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 42827 } }
I20260807 10:53:20.590382 22704 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa18b7264c0347678a0cc0a30c4fdf03, State: Initialized, Role: FOLLOWER
I20260807 10:53:20.591138 22704 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa18b7264c0347678a0cc0a30c4fdf03" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 42827 } }
I20260807 10:53:20.591444 22704 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:53:20.591590 22704 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:53:20.591789 22704 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03 [term 1 FOLLOWER]: Advancing to term 2
I20260807 10:53:20.596443 22704 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa18b7264c0347678a0cc0a30c4fdf03" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 42827 } }
I20260807 10:53:20.597123 22704 leader_election.cc:302] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa18b7264c0347678a0cc0a30c4fdf03; no voters:
I20260807 10:53:20.597784 22704 leader_election.cc:288] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 10:53:20.598028 22709 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 10:53:20.599929 22704 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:53:20.601063 22709 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03 [term 2 LEADER]: Becoming Leader. State: Replica: aa18b7264c0347678a0cc0a30c4fdf03, State: Running, Role: LEADER
I20260807 10:53:20.601816 22709 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa18b7264c0347678a0cc0a30c4fdf03" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 42827 } }
I20260807 10:53:20.605443 22711 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa18b7264c0347678a0cc0a30c4fdf03. Latest consensus state: current_term: 2 leader_uuid: "aa18b7264c0347678a0cc0a30c4fdf03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa18b7264c0347678a0cc0a30c4fdf03" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 42827 } } }
I20260807 10:53:20.605376 22710 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "aa18b7264c0347678a0cc0a30c4fdf03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa18b7264c0347678a0cc0a30c4fdf03" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 42827 } } }
I20260807 10:53:20.605957 22711 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03 [sys.catalog]: This master's current role is: LEADER
I20260807 10:53:20.606066 22710 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03 [sys.catalog]: This master's current role is: LEADER
I20260807 10:53:20.615731 22717 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:53:20.623366 22717 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:53:20.625523 22717 catalog_manager.cc:1259] Loaded cluster ID: ef07b1f345c346fd891b9aaec05bd787
I20260807 10:53:20.625666 22717 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:53:20.631122 22717 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:53:20.633455 22717 catalog_manager.cc:5879] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03: Loaded TSK: 0
I20260807 10:53:20.635521 22717 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 10:53:20.639761 22704 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260807 10:53:21.058241 22707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:53:21.058699 22707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:53:21.058931 22707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:53:21.071595 22707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:53:21.071790 22707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:53:21.072062 22707 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.193
I20260807 10:53:21.089231 22707 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:39741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.3.73.212:43333/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.193:42013
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.73.193
--webserver_port=46875
--tserver_master_addrs=127.3.73.254:42827
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.193
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:53:21.091171 22707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:53:21.093310 22707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:53:21.153513 22766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:21.155728 22765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:21.157671 22768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:53:21.158921 22707 server_base.cc:1034] running on GCE node
I20260807 10:53:21.160127 22707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:53:21.162130 22707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:53:21.165635 22707 hybrid_clock.cc:648] HybridClock initialized: now 1786100001165531 us; error 81 us; skew 500 ppm
I20260807 10:53:21.166114 22707 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:53:21 dist-test-slave-x5fp krb5kdc[21056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100001, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:53:21.171404 22707 init.cc:377] Logged in from keytab as kudu/127.3.73.193@KRBTEST.COM (short username kudu)
I20260807 10:53:21.182387 22707 webserver.cc:460] Webserver started at http://127.3.73.193:46875/ using document root <none> and password file <none>
I20260807 10:53:21.183786 22707 fs_manager.cc:359] Metadata directory not provided
I20260807 10:53:21.184118 22707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 10:53:21 dist-test-slave-x5fp krb5kdc[21056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100001, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for HTTP/127.3.73.212@KRBTEST.COM
I20260807 10:53:21.243860 22707 fs_manager.cc:709] Time spent opening directory manager: real 0.014s user 0.004s sys 0.008s
I20260807 10:53:21.254657 22780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:21.256872 22707 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.007s sys 0.004s
I20260807 10:53:21.257195 22707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-0/wal
uuid: "c96cd230adcd495e8d31a810c75f26a4"
format_stamp: "Formatted at 2026-08-07 10:53:17 on dist-test-slave-x5fp"
server_key: "e2fddb4615fd6f54e755a5436eff3c38"
server_key_iv: "9064bea98f0a4e58944fd0016a46550d"
server_key_version: "kuduclusterkey@0"
I20260807 10:53:21.258858 22707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:53:21.301898 22707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:53:21.307281 22707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:53:21.307626 22707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:53:21.308974 22707 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:53:21.310853 22707 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:53:21.310983 22707 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:21.311131 22707 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:53:21.311234 22707 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:21.370342 22707 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.193:42013
I20260807 10:53:21.370410 22956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.193:42013 every 8 connection(s)
I20260807 10:53:21.372766 22707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
I20260807 10:53:21.373944 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 22707
I20260807 10:53:21.374945 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.73.212:43333/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.194:34623
--local_ip_for_outbound_sockets=127.3.73.194
--tserver_master_addrs=127.3.73.254:42827
--webserver_port=44129
--webserver_interface=127.3.73.194
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:39741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.3.73.212:43333/kms with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 10:53:21 dist-test-slave-x5fp krb5kdc[21056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100001, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:53:21.426105 22963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:21.377675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:42827 (local address 127.3.73.193:53241)
0807 10:53:21.389559 (+ 11884us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:21.389626 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 10:53:21.395644 (+ 6018us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:53:21.398610 (+ 2966us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:53:21.398623 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:53:21.399605 (+ 982us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:53:21.401158 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:21.401199 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:21.404009 (+ 2810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:21.404017 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:21.406330 (+ 2313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:21.406357 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:21.408408 (+ 2051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:21.408419 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:21.408799 (+ 380us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:21.409744 (+ 945us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:53:21.409783 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:53:21.412894 (+ 3111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:53:21.418230 (+ 5336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:21.418246 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:21.418268 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:21.419117 (+ 849us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:21.419903 (+ 786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:21.419911 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:21.419917 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:21.420210 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:21.421375 (+ 1165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:53:21.421387 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:53:21.422180 (+ 793us) client_negotiation.cc:769] Sending connection context
0807 10:53:21.422575 (+ 395us) client_negotiation.cc:240] Negotiation successful
0807 10:53:21.424661 (+ 2086us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11442,"thread_start_us":11118,"threads_started":1}
I20260807 10:53:21.427561 22964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:21.379138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:53241 (local address 127.3.73.254:42827)
0807 10:53:21.393376 (+ 14238us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:21.393403 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:21.393480 (+ 77us) server_negotiation.cc:407] Connection header received
0807 10:53:21.396598 (+ 3118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:21.396622 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:21.397489 (+ 867us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:21.398307 (+ 818us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:21.401522 (+ 3215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:21.403658 (+ 2136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:21.406607 (+ 2949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:21.408050 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:21.408347 (+ 297us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:21.413325 (+ 4978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:21.413339 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:21.413373 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:21.413443 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:21.417968 (+ 4525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:21.419372 (+ 1404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:21.419386 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:21.419396 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:21.419684 (+ 288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:21.420381 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:21.420388 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:21.420391 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:21.421182 (+ 791us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:21.421435 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:21.423133 (+ 1698us) server_negotiation.cc:299] Negotiation successful
0807 10:53:21.423614 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13916,"thread_start_us":13670,"threads_started":1}
I20260807 10:53:21.432402 22957 heartbeater.cc:347] Connected to a master server at 127.3.73.254:42827
I20260807 10:53:21.432950 22957 heartbeater.cc:464] Registering TS with master...
I20260807 10:53:21.433940 22957 heartbeater.cc:511] Master 127.3.73.254:42827 requested a full tablet report, sending...
W20260807 10:53:21.438376 22957 heartbeater.cc:644] Failed to heartbeat to 127.3.73.254:42827 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 10:53:21.438639 22957 heartbeater.cc:464] Registering TS with master...
I20260807 10:53:21.438946 22957 heartbeater.cc:511] Master 127.3.73.254:42827 requested a full tablet report, sending...
I20260807 10:53:21.440650 22957 heartbeater.cc:464] Registering TS with master...
I20260807 10:53:21.441013 22957 heartbeater.cc:511] Master 127.3.73.254:42827 requested a full tablet report, sending...
W20260807 10:53:21.442368 22957 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 10:53:21.909101 22966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:53:21.909579 22966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:53:21.909803 22966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:53:21.920811 22966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:53:21.921131 22966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:53:21.921341 22966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.194
I20260807 10:53:21.934787 22966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:39741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.3.73.212:43333/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.194:34623
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.73.194
--webserver_port=44129
--tserver_master_addrs=127.3.73.254:42827
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.194
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:53:21.936998 22966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:53:21.938968 22966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:53:21.980510 22984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:21.982075 22988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:53:21.983572 22966 server_base.cc:1034] running on GCE node
W20260807 10:53:21.983771 22992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:53:21.984683 22966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:53:22.001149 22966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:53:22.002529 22966 hybrid_clock.cc:648] HybridClock initialized: now 1786100002002441 us; error 98 us; skew 500 ppm
I20260807 10:53:22.003428 22966 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:53:22 dist-test-slave-x5fp krb5kdc[21056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100002, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:53:22.008271 22966 init.cc:377] Logged in from keytab as kudu/127.3.73.194@KRBTEST.COM (short username kudu)
I20260807 10:53:22.013381 22966 webserver.cc:460] Webserver started at http://127.3.73.194:44129/ using document root <none> and password file <none>
I20260807 10:53:22.014546 22966 fs_manager.cc:359] Metadata directory not provided
I20260807 10:53:22.014742 22966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 10:53:22 dist-test-slave-x5fp krb5kdc[21056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100002, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for HTTP/127.3.73.212@KRBTEST.COM
I20260807 10:53:22.050828 22966 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.009s sys 0.000s
I20260807 10:53:22.055270 23006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:22.057384 22966 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 10:53:22.057641 22966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-1/wal
uuid: "d81c4bbc01254e8eaee0c19e0d3d775f"
format_stamp: "Formatted at 2026-08-07 10:53:18 on dist-test-slave-x5fp"
server_key: "46592f31c22d635ad5b83e6ef08f1e7b"
server_key_iv: "217437aec59884c4a4eb4532b76ec4d5"
server_key_version: "kuduclusterkey@0"
I20260807 10:53:22.059240 22966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 07 10:53:22 dist-test-slave-x5fp krb5kdc[21056](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786100002, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:53:22.085589 22966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:53:22.091400 22966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:53:22.091881 22966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:53:22.095435 22966 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:53:22.101599 22966 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:53:22.101763 22966 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:22.101979 22966 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:53:22.102111 22966 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:22.164446 22966 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.194:34623
I20260807 10:53:22.164480 23176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.194:34623 every 8 connection(s)
I20260807 10:53:22.166528 22966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
I20260807 10:53:22.175855 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 22966
I20260807 10:53:22.176898 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.73.212:43333/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.195:43601
--local_ip_for_outbound_sockets=127.3.73.195
--tserver_master_addrs=127.3.73.254:42827
--webserver_port=37239
--webserver_interface=127.3.73.195
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:39741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.3.73.212:43333/kms with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 10:53:22 dist-test-slave-x5fp krb5kdc[21056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100002, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:53:22.196393 23181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:22.169877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:42827 (local address 127.3.73.194:40021)
0807 10:53:22.171658 (+ 1781us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:22.171729 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0807 10:53:22.173816 (+ 2087us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:53:22.175420 (+ 1604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:53:22.175430 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:53:22.175952 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:53:22.176993 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:22.177040 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:22.178407 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:22.178413 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:22.179972 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:22.180002 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:22.181744 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:22.181752 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:22.182093 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:22.182993 (+ 900us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:53:22.183029 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:53:22.185847 (+ 2818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:53:22.190428 (+ 4581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:22.190444 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:22.190464 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:22.191192 (+ 728us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:22.191783 (+ 591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:22.191789 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:22.191794 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:22.192102 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:22.193113 (+ 1011us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:53:22.193123 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:53:22.193637 (+ 514us) client_negotiation.cc:769] Sending connection context
0807 10:53:22.194013 (+ 376us) client_negotiation.cc:240] Negotiation successful
0807 10:53:22.194702 (+ 689us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":733,"threads_started":1}
I20260807 10:53:22.197984 23182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:22.170553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:40021 (local address 127.3.73.254:42827)
0807 10:53:22.171635 (+ 1082us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:22.171643 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:22.172302 (+ 659us) server_negotiation.cc:407] Connection header received
0807 10:53:22.174821 (+ 2519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:22.174828 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:22.174948 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:22.175086 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:22.177279 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:22.178138 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:22.180197 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:22.181238 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:22.181377 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:22.186176 (+ 4799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:22.186185 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:22.186190 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:22.186237 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:22.190199 (+ 3962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:22.191380 (+ 1181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:22.191388 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:22.191393 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:22.191591 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:22.192388 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:22.192394 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:22.192398 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:22.192926 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:22.197007 (+ 4081us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:22.197241 (+ 234us) server_negotiation.cc:299] Negotiation successful
0807 10:53:22.197540 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":942,"thread_start_us":1002,"threads_started":1}
I20260807 10:53:22.199739 23177 heartbeater.cc:347] Connected to a master server at 127.3.73.254:42827
I20260807 10:53:22.200188 23177 heartbeater.cc:464] Registering TS with master...
I20260807 10:53:22.201401 23177 heartbeater.cc:511] Master 127.3.73.254:42827 requested a full tablet report, sending...
I20260807 10:53:22.204015 22649 ts_manager.cc:194] Registered new tserver with Master: d81c4bbc01254e8eaee0c19e0d3d775f (127.3.73.194:34623)
I20260807 10:53:22.206564 22649 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.194@KRBTEST.COM'} at 127.3.73.194:40021
I20260807 10:53:22.442584 22957 heartbeater.cc:464] Registering TS with master...
I20260807 10:53:22.443109 22957 heartbeater.cc:511] Master 127.3.73.254:42827 requested a full tablet report, sending...
I20260807 10:53:22.444754 22649 ts_manager.cc:194] Registered new tserver with Master: c96cd230adcd495e8d31a810c75f26a4 (127.3.73.193:42013)
I20260807 10:53:22.446095 22649 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.193@KRBTEST.COM'} at 127.3.73.193:53241
Aug 07 10:53:22 dist-test-slave-x5fp krb5kdc[21056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100002, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for HTTP/127.3.73.212@KRBTEST.COM
W20260807 10:53:22.701341 23185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:53:22.701978 23185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:53:22.702193 23185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:53:22.715461 23185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:53:22.715780 23185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:53:22.716029 23185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.195
I20260807 10:53:22.732066 23185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:39741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.3.73.212:43333/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.195:43601
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.73.195
--webserver_port=37239
--tserver_master_addrs=127.3.73.254:42827
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.195
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:53:22.733989 23185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:53:22.735720 23185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:53:22.753502 23192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:22.753561 23191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:22.753908 23195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:53:22.755535 23185 server_base.cc:1034] running on GCE node
I20260807 10:53:22.756421 23185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:53:22.757761 23185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:53:22.759037 23185 hybrid_clock.cc:648] HybridClock initialized: now 1786100002758973 us; error 43 us; skew 500 ppm
I20260807 10:53:22.759505 23185 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:53:22 dist-test-slave-x5fp krb5kdc[21056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100002, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:53:22.764226 23185 init.cc:377] Logged in from keytab as kudu/127.3.73.195@KRBTEST.COM (short username kudu)
I20260807 10:53:22.766899 23185 webserver.cc:460] Webserver started at http://127.3.73.195:37239/ using document root <none> and password file <none>
I20260807 10:53:22.768045 23185 fs_manager.cc:359] Metadata directory not provided
I20260807 10:53:22.768281 23185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 10:53:22 dist-test-slave-x5fp krb5kdc[21056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100002, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for HTTP/127.3.73.212@KRBTEST.COM
I20260807 10:53:22.807370 23185 fs_manager.cc:709] Time spent opening directory manager: real 0.014s user 0.008s sys 0.004s
I20260807 10:53:22.811932 23209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:22.814052 23185 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 10:53:22.814327 23185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-2/wal
uuid: "4209b5f7d4d34c82bb07dcb0b78463d3"
format_stamp: "Formatted at 2026-08-07 10:53:19 on dist-test-slave-x5fp"
server_key: "ecea9bd167d6ed4c44a67767427aa1b6"
server_key_iv: "8d0be2262a6fe01e186cc556213bf701"
server_key_version: "kuduclusterkey@0"
I20260807 10:53:22.815948 23185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:53:22.836963 23185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:53:22.842944 23185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:53:22.843325 23185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:53:22.844803 23185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:53:22.846976 23185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:53:22.847100 23185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:22.847218 23185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:53:22.847307 23185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:22.902277 23185 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.195:43601
I20260807 10:53:22.902333 23375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.195:43601 every 8 connection(s)
I20260807 10:53:22.904618 23185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
I20260807 10:53:22.907912 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 23185
Aug 07 10:53:22 dist-test-slave-x5fp krb5kdc[21056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100002, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:53:22.980069 23385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:22.943839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40396 (local address 127.3.73.254:42827)
0807 10:53:22.950592 (+ 6753us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:22.950599 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:22.950620 (+ 21us) server_negotiation.cc:407] Connection header received
0807 10:53:22.950677 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:22.950684 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:22.950795 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:22.953495 (+ 2700us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:22.953747 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:22.958006 (+ 4259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:22.962276 (+ 4270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:22.962988 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:22.963129 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:22.967736 (+ 4607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:22.967744 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:22.967749 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:22.967808 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:22.975976 (+ 8168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:22.977281 (+ 1305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:22.977289 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:22.977293 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:22.977486 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:22.977984 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:22.977990 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:22.977994 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:22.978401 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:22.978700 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:22.979370 (+ 670us) server_negotiation.cc:299] Negotiation successful
0807 10:53:22.979659 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6599,"thread_start_us":7187,"threads_started":1}
I20260807 10:53:22.987808 23381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:22.910167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:45789 (local address 127.3.73.254:42827)
0807 10:53:22.943347 (+ 33180us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:22.943354 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:22.949358 (+ 6004us) server_negotiation.cc:407] Connection header received
0807 10:53:22.949455 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:22.949461 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:22.949589 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:22.949729 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:22.959153 (+ 9424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:22.960469 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:22.963273 (+ 2804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:22.963991 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:22.964105 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:22.971953 (+ 7848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:22.971962 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:22.971967 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:22.972061 (+ 94us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:22.981408 (+ 9347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:22.982567 (+ 1159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:22.982574 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:22.982577 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:22.983101 (+ 524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:22.985206 (+ 2105us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:22.985212 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:22.985216 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:22.985607 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:22.985760 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:22.987091 (+ 1331us) server_negotiation.cc:299] Negotiation successful
0807 10:53:22.987394 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32999,"thread_start_us":33442,"threads_started":1}
I20260807 10:53:22.990599 23379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:22.908580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:42827 (local address 127.3.73.195:45789)
0807 10:53:22.944820 (+ 36240us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:22.944905 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0807 10:53:22.947712 (+ 2807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:53:22.955328 (+ 7616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:53:22.955346 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:53:22.956171 (+ 825us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:53:22.958810 (+ 2639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:22.958875 (+ 65us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:22.960769 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:22.960775 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:22.963049 (+ 2274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:22.963081 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:22.964508 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:22.964515 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:22.964888 (+ 373us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:22.967810 (+ 2922us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:53:22.967847 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:53:22.971559 (+ 3712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:53:22.981632 (+ 10073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:22.981645 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:22.981668 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:22.982406 (+ 738us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:22.983293 (+ 887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:22.983298 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:22.983301 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:22.983652 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:22.985906 (+ 2254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:53:22.985916 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:53:22.986449 (+ 533us) client_negotiation.cc:769] Sending connection context
0807 10:53:22.988086 (+ 1637us) client_negotiation.cc:240] Negotiation successful
0807 10:53:22.988842 (+ 756us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35786,"thread_start_us":39515,"threads_started":1}
I20260807 10:53:22.993058 23377 heartbeater.cc:347] Connected to a master server at 127.3.73.254:42827
I20260807 10:53:22.993424 23377 heartbeater.cc:464] Registering TS with master...
I20260807 10:53:22.994426 23377 heartbeater.cc:511] Master 127.3.73.254:42827 requested a full tablet report, sending...
I20260807 10:53:22.998157 22648 ts_manager.cc:194] Registered new tserver with Master: 4209b5f7d4d34c82bb07dcb0b78463d3 (127.3.73.195:43601)
I20260807 10:53:22.999599 22648 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.195@KRBTEST.COM'} at 127.3.73.195:45789
I20260807 10:53:23.001752 3367 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 10:53:23 dist-test-slave-x5fp krb5kdc[21056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100002, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for HTTP/127.3.73.212@KRBTEST.COM
I20260807 10:53:23.041899 23381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:23.022938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40408 (local address 127.3.73.254:42827)
0807 10:53:23.023307 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:23.023315 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:23.024553 (+ 1238us) server_negotiation.cc:407] Connection header received
0807 10:53:23.024812 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:23.024819 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:23.024926 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:23.025067 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:23.027124 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:23.028245 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:23.029963 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:23.030526 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:23.030694 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:23.034554 (+ 3860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:23.034560 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:23.034564 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:23.034607 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:23.038297 (+ 3690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:23.039417 (+ 1120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:23.039424 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:23.039427 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:23.039620 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:23.040107 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:23.040113 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:23.040116 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:23.040532 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:23.040837 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:23.041276 (+ 439us) server_negotiation.cc:299] Negotiation successful
0807 10:53:23.041551 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202}
I20260807 10:53:23.050172 22649 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40408:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 10:53:23.060761 22649 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 10:53:23.136123 23405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:23.104331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:34623 (local address 127.0.0.1:33564)
0807 10:53:23.125290 (+ 20959us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:23.125318 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 10:53:23.125484 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:53:23.126719 (+ 1235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:53:23.126728 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:53:23.126766 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:53:23.127233 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:23.127251 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:23.129432 (+ 2181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:23.129439 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:23.131926 (+ 2487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:23.131951 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:23.135011 (+ 3060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:23.135018 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:23.135252 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:23.135318 (+ 66us) client_negotiation.cc:769] Sending connection context
0807 10:53:23.135478 (+ 160us) client_negotiation.cc:240] Negotiation successful
0807 10:53:23.135709 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20797,"thread_start_us":20641,"threads_started":1}
I20260807 10:53:23.137394 23406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:23.104570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33564 (local address 127.3.73.194:34623)
0807 10:53:23.121285 (+ 16715us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:23.121293 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:23.125643 (+ 4350us) server_negotiation.cc:407] Connection header received
0807 10:53:23.125778 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:23.125787 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:23.126080 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:23.126395 (+ 315us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:53:23.127473 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:23.129219 (+ 1746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:23.132179 (+ 2960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:23.134387 (+ 2208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:23.134513 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:23.134737 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:23.136837 (+ 2100us) server_negotiation.cc:299] Negotiation successful
0807 10:53:23.137044 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16558,"thread_start_us":16401,"threads_started":1}
I20260807 10:53:23.143157 23402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:23.104054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:42013 (local address 127.0.0.1:55256)
0807 10:53:23.120103 (+ 16049us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:23.120184 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0807 10:53:23.120527 (+ 343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:53:23.131393 (+ 10866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:53:23.131402 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:53:23.131468 (+ 66us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:53:23.132037 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:23.132053 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:23.136156 (+ 4103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:23.136165 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:23.139612 (+ 3447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:23.139639 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:23.142242 (+ 2603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:23.142251 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:23.142448 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:23.142487 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 10:53:23.142585 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 10:53:23.142742 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15663,"thread_start_us":15453,"threads_started":1}
I20260807 10:53:23.144047 23408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:23.104825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55256 (local address 127.3.73.193:42013)
0807 10:53:23.119262 (+ 14437us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:23.119279 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:23.120861 (+ 1582us) server_negotiation.cc:407] Connection header received
0807 10:53:23.121011 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:23.121024 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:23.121387 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:23.125013 (+ 3626us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:53:23.132319 (+ 7306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:23.135918 (+ 3599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:23.139900 (+ 3982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:23.141548 (+ 1648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:23.141728 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:23.142029 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:23.143468 (+ 1439us) server_negotiation.cc:299] Negotiation successful
0807 10:53:23.143679 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14136,"thread_start_us":13859,"threads_started":1}
I20260807 10:53:23.148977 23077 tablet_service.cc:1467] Processing CreateTablet for tablet 6d0667dcb3964557ad1862ddc504f2e0 (DEFAULT_TABLE table=test-table [id=eeec37ade68047459ff59a893001fed0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:53:23.149463 22853 tablet_service.cc:1467] Processing CreateTablet for tablet 6d0667dcb3964557ad1862ddc504f2e0 (DEFAULT_TABLE table=test-table [id=eeec37ade68047459ff59a893001fed0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:53:23.150933 23077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d0667dcb3964557ad1862ddc504f2e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:53:23.154579 22853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d0667dcb3964557ad1862ddc504f2e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:53:23.186633 23405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:23.132234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:43601 (local address 127.0.0.1:37200)
0807 10:53:23.136459 (+ 4225us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:23.136481 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 10:53:23.136607 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:53:23.174388 (+ 37781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:53:23.174396 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:53:23.174437 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:53:23.174643 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:23.174658 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:23.178844 (+ 4186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:23.178853 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:23.180709 (+ 1856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:23.180773 (+ 64us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:23.185410 (+ 4637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:23.185422 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:23.185733 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:23.185799 (+ 66us) client_negotiation.cc:769] Sending connection context
0807 10:53:23.185907 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 10:53:23.186148 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4096,"thread_start_us":9691,"threads_started":1}
I20260807 10:53:23.187849 23413 tablet_bootstrap.cc:492] T 6d0667dcb3964557ad1862ddc504f2e0 P d81c4bbc01254e8eaee0c19e0d3d775f: Bootstrap starting.
I20260807 10:53:23.188454 23411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:23.126006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37200 (local address 127.3.73.195:43601)
0807 10:53:23.167319 (+ 41313us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:23.167329 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:23.167372 (+ 43us) server_negotiation.cc:407] Connection header received
0807 10:53:23.167488 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:23.167496 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:23.167813 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:23.174302 (+ 6489us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:53:23.176821 (+ 2519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:23.178615 (+ 1794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:23.181009 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:23.182744 (+ 1735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:23.187288 (+ 4544us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:23.187703 (+ 415us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:23.187809 (+ 106us) server_negotiation.cc:299] Negotiation successful
0807 10:53:23.188073 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41100,"thread_start_us":41985,"threads_started":1}
I20260807 10:53:23.192142 23414 tablet_bootstrap.cc:492] T 6d0667dcb3964557ad1862ddc504f2e0 P c96cd230adcd495e8d31a810c75f26a4: Bootstrap starting.
I20260807 10:53:23.194053 23413 tablet_bootstrap.cc:654] T 6d0667dcb3964557ad1862ddc504f2e0 P d81c4bbc01254e8eaee0c19e0d3d775f: Neither blocks nor log segments found. Creating new log.
I20260807 10:53:23.193735 23269 tablet_service.cc:1467] Processing CreateTablet for tablet 6d0667dcb3964557ad1862ddc504f2e0 (DEFAULT_TABLE table=test-table [id=eeec37ade68047459ff59a893001fed0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:53:23.195603 23413 log.cc:824] T 6d0667dcb3964557ad1862ddc504f2e0 P d81c4bbc01254e8eaee0c19e0d3d775f: Log is configured to *not* fsync() on all Append() calls
I20260807 10:53:23.197002 23269 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d0667dcb3964557ad1862ddc504f2e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:53:23.211833 23414 tablet_bootstrap.cc:654] T 6d0667dcb3964557ad1862ddc504f2e0 P c96cd230adcd495e8d31a810c75f26a4: Neither blocks nor log segments found. Creating new log.
I20260807 10:53:23.212539 23177 heartbeater.cc:503] Master 127.3.73.254:42827 was elected leader, sending a full tablet report...
I20260807 10:53:23.213986 23414 log.cc:824] T 6d0667dcb3964557ad1862ddc504f2e0 P c96cd230adcd495e8d31a810c75f26a4: Log is configured to *not* fsync() on all Append() calls
I20260807 10:53:23.226799 23414 tablet_bootstrap.cc:492] T 6d0667dcb3964557ad1862ddc504f2e0 P c96cd230adcd495e8d31a810c75f26a4: No bootstrap required, opened a new log
I20260807 10:53:23.227242 23414 ts_tablet_manager.cc:1397] T 6d0667dcb3964557ad1862ddc504f2e0 P c96cd230adcd495e8d31a810c75f26a4: Time spent bootstrapping tablet: real 0.035s user 0.012s sys 0.000s
I20260807 10:53:23.227697 23413 tablet_bootstrap.cc:492] T 6d0667dcb3964557ad1862ddc504f2e0 P d81c4bbc01254e8eaee0c19e0d3d775f: No bootstrap required, opened a new log
I20260807 10:53:23.228039 23413 ts_tablet_manager.cc:1397] T 6d0667dcb3964557ad1862ddc504f2e0 P d81c4bbc01254e8eaee0c19e0d3d775f: Time spent bootstrapping tablet: real 0.041s user 0.015s sys 0.019s
I20260807 10:53:23.236977 23418 tablet_bootstrap.cc:492] T 6d0667dcb3964557ad1862ddc504f2e0 P 4209b5f7d4d34c82bb07dcb0b78463d3: Bootstrap starting.
I20260807 10:53:23.239193 23414 raft_consensus.cc:348] T 6d0667dcb3964557ad1862ddc504f2e0 P c96cd230adcd495e8d31a810c75f26a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96cd230adcd495e8d31a810c75f26a4" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 42013 } } peers { permanent_uuid: "d81c4bbc01254e8eaee0c19e0d3d775f" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34623 } } peers { permanent_uuid: "4209b5f7d4d34c82bb07dcb0b78463d3" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 43601 } }
I20260807 10:53:23.239965 23414 raft_consensus.cc:374] T 6d0667dcb3964557ad1862ddc504f2e0 P c96cd230adcd495e8d31a810c75f26a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:53:23.240146 23414 raft_consensus.cc:729] T 6d0667dcb3964557ad1862ddc504f2e0 P c96cd230adcd495e8d31a810c75f26a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c96cd230adcd495e8d31a810c75f26a4, State: Initialized, Role: FOLLOWER
I20260807 10:53:23.241114 23414 consensus_queue.cc:260] T 6d0667dcb3964557ad1862ddc504f2e0 P c96cd230adcd495e8d31a810c75f26a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96cd230adcd495e8d31a810c75f26a4" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 42013 } } peers { permanent_uuid: "d81c4bbc01254e8eaee0c19e0d3d775f" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34623 } } peers { permanent_uuid: "4209b5f7d4d34c82bb07dcb0b78463d3" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 43601 } }
I20260807 10:53:23.245307 23413 raft_consensus.cc:348] T 6d0667dcb3964557ad1862ddc504f2e0 P d81c4bbc01254e8eaee0c19e0d3d775f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96cd230adcd495e8d31a810c75f26a4" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 42013 } } peers { permanent_uuid: "d81c4bbc01254e8eaee0c19e0d3d775f" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34623 } } peers { permanent_uuid: "4209b5f7d4d34c82bb07dcb0b78463d3" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 43601 } }
I20260807 10:53:23.245990 23413 raft_consensus.cc:374] T 6d0667dcb3964557ad1862ddc504f2e0 P d81c4bbc01254e8eaee0c19e0d3d775f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:53:23.246116 23413 raft_consensus.cc:729] T 6d0667dcb3964557ad1862ddc504f2e0 P d81c4bbc01254e8eaee0c19e0d3d775f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d81c4bbc01254e8eaee0c19e0d3d775f, State: Initialized, Role: FOLLOWER
I20260807 10:53:23.247028 23418 tablet_bootstrap.cc:654] T 6d0667dcb3964557ad1862ddc504f2e0 P 4209b5f7d4d34c82bb07dcb0b78463d3: Neither blocks nor log segments found. Creating new log.
I20260807 10:53:23.248910 23418 log.cc:824] T 6d0667dcb3964557ad1862ddc504f2e0 P 4209b5f7d4d34c82bb07dcb0b78463d3: Log is configured to *not* fsync() on all Append() calls
I20260807 10:53:23.250559 23414 ts_tablet_manager.cc:1428] T 6d0667dcb3964557ad1862ddc504f2e0 P c96cd230adcd495e8d31a810c75f26a4: Time spent starting tablet: real 0.023s user 0.017s sys 0.004s
I20260807 10:53:23.251446 23413 consensus_queue.cc:260] T 6d0667dcb3964557ad1862ddc504f2e0 P d81c4bbc01254e8eaee0c19e0d3d775f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96cd230adcd495e8d31a810c75f26a4" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 42013 } } peers { permanent_uuid: "d81c4bbc01254e8eaee0c19e0d3d775f" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34623 } } peers { permanent_uuid: "4209b5f7d4d34c82bb07dcb0b78463d3" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 43601 } }
I20260807 10:53:23.252691 22957 heartbeater.cc:503] Master 127.3.73.254:42827 was elected leader, sending a full tablet report...
I20260807 10:53:23.260006 23413 ts_tablet_manager.cc:1428] T 6d0667dcb3964557ad1862ddc504f2e0 P d81c4bbc01254e8eaee0c19e0d3d775f: Time spent starting tablet: real 0.032s user 0.009s sys 0.009s
I20260807 10:53:23.264021 23418 tablet_bootstrap.cc:492] T 6d0667dcb3964557ad1862ddc504f2e0 P 4209b5f7d4d34c82bb07dcb0b78463d3: No bootstrap required, opened a new log
I20260807 10:53:23.264384 23418 ts_tablet_manager.cc:1397] T 6d0667dcb3964557ad1862ddc504f2e0 P 4209b5f7d4d34c82bb07dcb0b78463d3: Time spent bootstrapping tablet: real 0.039s user 0.009s sys 0.005s
I20260807 10:53:23.275817 23418 raft_consensus.cc:348] T 6d0667dcb3964557ad1862ddc504f2e0 P 4209b5f7d4d34c82bb07dcb0b78463d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96cd230adcd495e8d31a810c75f26a4" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 42013 } } peers { permanent_uuid: "d81c4bbc01254e8eaee0c19e0d3d775f" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34623 } } peers { permanent_uuid: "4209b5f7d4d34c82bb07dcb0b78463d3" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 43601 } }
I20260807 10:53:23.276357 23418 raft_consensus.cc:374] T 6d0667dcb3964557ad1862ddc504f2e0 P 4209b5f7d4d34c82bb07dcb0b78463d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:53:23.276490 23418 raft_consensus.cc:729] T 6d0667dcb3964557ad1862ddc504f2e0 P 4209b5f7d4d34c82bb07dcb0b78463d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4209b5f7d4d34c82bb07dcb0b78463d3, State: Initialized, Role: FOLLOWER
I20260807 10:53:23.277480 23418 consensus_queue.cc:260] T 6d0667dcb3964557ad1862ddc504f2e0 P 4209b5f7d4d34c82bb07dcb0b78463d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96cd230adcd495e8d31a810c75f26a4" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 42013 } } peers { permanent_uuid: "d81c4bbc01254e8eaee0c19e0d3d775f" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34623 } } peers { permanent_uuid: "4209b5f7d4d34c82bb07dcb0b78463d3" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 43601 } }
I20260807 10:53:23.278864 23377 heartbeater.cc:503] Master 127.3.73.254:42827 was elected leader, sending a full tablet report...
I20260807 10:53:23.279961 23418 ts_tablet_manager.cc:1428] T 6d0667dcb3964557ad1862ddc504f2e0 P 4209b5f7d4d34c82bb07dcb0b78463d3: Time spent starting tablet: real 0.015s user 0.011s sys 0.004s
I20260807 10:53:23.353440 23423 raft_consensus.cc:482] T 6d0667dcb3964557ad1862ddc504f2e0 P d81c4bbc01254e8eaee0c19e0d3d775f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 10:53:23.353833 23423 raft_consensus.cc:504] T 6d0667dcb3964557ad1862ddc504f2e0 P d81c4bbc01254e8eaee0c19e0d3d775f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96cd230adcd495e8d31a810c75f26a4" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 42013 } } peers { permanent_uuid: "d81c4bbc01254e8eaee0c19e0d3d775f" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34623 } } peers { permanent_uuid: "4209b5f7d4d34c82bb07dcb0b78463d3" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 43601 } }
I20260807 10:53:23.355768 23423 leader_election.cc:288] T 6d0667dcb3964557ad1862ddc504f2e0 P d81c4bbc01254e8eaee0c19e0d3d775f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c96cd230adcd495e8d31a810c75f26a4 (127.3.73.193:42013), 4209b5f7d4d34c82bb07dcb0b78463d3 (127.3.73.195:43601)
I20260807 10:53:23.365796 23411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:23.356595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:34839 (local address 127.3.73.195:43601)
0807 10:53:23.356833 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:23.356840 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:23.358076 (+ 1236us) server_negotiation.cc:407] Connection header received
0807 10:53:23.358406 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:23.358413 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:23.358506 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:23.358586 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:53:23.359631 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:23.361009 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:23.363523 (+ 2514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:23.364690 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:23.364928 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:23.365015 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:23.365326 (+ 311us) server_negotiation.cc:299] Negotiation successful
0807 10:53:23.365502 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 10:53:23.366120 23436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:23.356445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:43601 (local address 127.3.73.194:34839)
0807 10:53:23.357876 (+ 1431us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:23.357899 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 10:53:23.358147 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:53:23.358801 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:53:23.358806 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:53:23.358837 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:53:23.359293 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:23.359368 (+ 75us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:23.361195 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:23.361201 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:23.363308 (+ 2107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:23.363327 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:23.364954 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:23.364960 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:23.365126 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:23.365159 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 10:53:23.365269 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 10:53:23.365505 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1272,"mutex_wait_us":86,"thread_start_us":1157,"threads_started":1}
I20260807 10:53:23.366777 23408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:23.356573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:57911 (local address 127.3.73.193:42013)
0807 10:53:23.356854 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:23.356863 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:23.358828 (+ 1965us) server_negotiation.cc:407] Connection header received
0807 10:53:23.359055 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:23.359062 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:23.359164 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:23.359273 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:53:23.360021 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:23.361258 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:23.363964 (+ 2706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:23.365396 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:23.365554 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:23.365628 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:23.366311 (+ 683us) server_negotiation.cc:299] Negotiation successful
0807 10:53:23.366466 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260807 10:53:23.367540 23437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:23.356677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:42013 (local address 127.3.73.194:57911)
0807 10:53:23.358660 (+ 1983us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:23.358681 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 10:53:23.358882 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:53:23.359318 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:53:23.359324 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:53:23.359353 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:53:23.359817 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:23.359830 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:23.361456 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:23.361461 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:23.363751 (+ 2290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:23.363767 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:23.365877 (+ 2110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:23.365886 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:23.366102 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:23.366143 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 10:53:23.366988 (+ 845us) client_negotiation.cc:240] Negotiation successful
0807 10:53:23.367147 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1859,"mutex_wait_us":53,"thread_start_us":1587,"threads_started":1}
I20260807 10:53:23.367790 23305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d0667dcb3964557ad1862ddc504f2e0" candidate_uuid: "d81c4bbc01254e8eaee0c19e0d3d775f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4209b5f7d4d34c82bb07dcb0b78463d3" is_pre_election: true
I20260807 10:53:23.368707 23305 raft_consensus.cc:2454] T 6d0667dcb3964557ad1862ddc504f2e0 P 4209b5f7d4d34c82bb07dcb0b78463d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d81c4bbc01254e8eaee0c19e0d3d775f in term 0.
I20260807 10:53:23.369248 22881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d0667dcb3964557ad1862ddc504f2e0" candidate_uuid: "d81c4bbc01254e8eaee0c19e0d3d775f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c96cd230adcd495e8d31a810c75f26a4" is_pre_election: true
I20260807 10:53:23.369637 23008 leader_election.cc:302] T 6d0667dcb3964557ad1862ddc504f2e0 P d81c4bbc01254e8eaee0c19e0d3d775f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4209b5f7d4d34c82bb07dcb0b78463d3, d81c4bbc01254e8eaee0c19e0d3d775f; no voters:
I20260807 10:53:23.369954 22881 raft_consensus.cc:2454] T 6d0667dcb3964557ad1862ddc504f2e0 P c96cd230adcd495e8d31a810c75f26a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d81c4bbc01254e8eaee0c19e0d3d775f in term 0.
I20260807 10:53:23.370167 23423 raft_consensus.cc:2781] T 6d0667dcb3964557ad1862ddc504f2e0 P d81c4bbc01254e8eaee0c19e0d3d775f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 10:53:23.370307 23423 raft_consensus.cc:482] T 6d0667dcb3964557ad1862ddc504f2e0 P d81c4bbc01254e8eaee0c19e0d3d775f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 10:53:23.370414 23423 raft_consensus.cc:3037] T 6d0667dcb3964557ad1862ddc504f2e0 P d81c4bbc01254e8eaee0c19e0d3d775f [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:53:23.374188 23423 raft_consensus.cc:504] T 6d0667dcb3964557ad1862ddc504f2e0 P d81c4bbc01254e8eaee0c19e0d3d775f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96cd230adcd495e8d31a810c75f26a4" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 42013 } } peers { permanent_uuid: "d81c4bbc01254e8eaee0c19e0d3d775f" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34623 } } peers { permanent_uuid: "4209b5f7d4d34c82bb07dcb0b78463d3" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 43601 } }
I20260807 10:53:23.375577 22881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d0667dcb3964557ad1862ddc504f2e0" candidate_uuid: "d81c4bbc01254e8eaee0c19e0d3d775f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c96cd230adcd495e8d31a810c75f26a4"
I20260807 10:53:23.375715 23305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d0667dcb3964557ad1862ddc504f2e0" candidate_uuid: "d81c4bbc01254e8eaee0c19e0d3d775f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4209b5f7d4d34c82bb07dcb0b78463d3"
I20260807 10:53:23.375829 22881 raft_consensus.cc:3037] T 6d0667dcb3964557ad1862ddc504f2e0 P c96cd230adcd495e8d31a810c75f26a4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:53:23.375950 23305 raft_consensus.cc:3037] T 6d0667dcb3964557ad1862ddc504f2e0 P 4209b5f7d4d34c82bb07dcb0b78463d3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:53:23.379297 22881 raft_consensus.cc:2454] T 6d0667dcb3964557ad1862ddc504f2e0 P c96cd230adcd495e8d31a810c75f26a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d81c4bbc01254e8eaee0c19e0d3d775f in term 1.
I20260807 10:53:23.379648 23305 raft_consensus.cc:2454] T 6d0667dcb3964557ad1862ddc504f2e0 P 4209b5f7d4d34c82bb07dcb0b78463d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d81c4bbc01254e8eaee0c19e0d3d775f in term 1.
I20260807 10:53:23.379873 23423 leader_election.cc:288] T 6d0667dcb3964557ad1862ddc504f2e0 P d81c4bbc01254e8eaee0c19e0d3d775f [CANDIDATE]: Term 1 election: Requested vote from peers c96cd230adcd495e8d31a810c75f26a4 (127.3.73.193:42013), 4209b5f7d4d34c82bb07dcb0b78463d3 (127.3.73.195:43601)
I20260807 10:53:23.380048 23007 leader_election.cc:302] T 6d0667dcb3964557ad1862ddc504f2e0 P d81c4bbc01254e8eaee0c19e0d3d775f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c96cd230adcd495e8d31a810c75f26a4, d81c4bbc01254e8eaee0c19e0d3d775f; no voters:
I20260807 10:53:23.380440 23423 raft_consensus.cc:2781] T 6d0667dcb3964557ad1862ddc504f2e0 P d81c4bbc01254e8eaee0c19e0d3d775f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:53:23.381136 23423 raft_consensus.cc:686] T 6d0667dcb3964557ad1862ddc504f2e0 P d81c4bbc01254e8eaee0c19e0d3d775f [term 1 LEADER]: Becoming Leader. State: Replica: d81c4bbc01254e8eaee0c19e0d3d775f, State: Running, Role: LEADER
I20260807 10:53:23.381783 23423 consensus_queue.cc:237] T 6d0667dcb3964557ad1862ddc504f2e0 P d81c4bbc01254e8eaee0c19e0d3d775f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96cd230adcd495e8d31a810c75f26a4" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 42013 } } peers { permanent_uuid: "d81c4bbc01254e8eaee0c19e0d3d775f" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34623 } } peers { permanent_uuid: "4209b5f7d4d34c82bb07dcb0b78463d3" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 43601 } }
W20260807 10:53:23.382211 22958 tablet.cc:2369] T 6d0667dcb3964557ad1862ddc504f2e0 P c96cd230adcd495e8d31a810c75f26a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:53:23.387705 22649 catalog_manager.cc:5495] T 6d0667dcb3964557ad1862ddc504f2e0 P d81c4bbc01254e8eaee0c19e0d3d775f reported cstate change: term changed from 0 to 1, leader changed from <none> to d81c4bbc01254e8eaee0c19e0d3d775f (127.3.73.194). New cstate: current_term: 1 leader_uuid: "d81c4bbc01254e8eaee0c19e0d3d775f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c96cd230adcd495e8d31a810c75f26a4" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 42013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d81c4bbc01254e8eaee0c19e0d3d775f" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 34623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4209b5f7d4d34c82bb07dcb0b78463d3" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 43601 } health_report { overall_health: UNKNOWN } } }
I20260807 10:53:23.423166 23406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:23.416998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33566 (local address 127.3.73.194:34623)
0807 10:53:23.417259 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:23.417265 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:23.417288 (+ 23us) server_negotiation.cc:407] Connection header received
0807 10:53:23.417514 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:23.417518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:23.417617 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:23.417827 (+ 210us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:53:23.418252 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:23.418989 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:23.420506 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:23.420998 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:23.421143 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:23.421814 (+ 671us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:53:23.422454 (+ 640us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:53:23.422658 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:23.422713 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 10:53:23.422804 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260807 10:53:23.423448 23178 tablet.cc:2369] T 6d0667dcb3964557ad1862ddc504f2e0 P d81c4bbc01254e8eaee0c19e0d3d775f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:53:23.433636 23305 raft_consensus.cc:1261] T 6d0667dcb3964557ad1862ddc504f2e0 P 4209b5f7d4d34c82bb07dcb0b78463d3 [term 1 FOLLOWER]: Refusing update from remote peer d81c4bbc01254e8eaee0c19e0d3d775f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:53:23.433878 22881 raft_consensus.cc:1261] T 6d0667dcb3964557ad1862ddc504f2e0 P c96cd230adcd495e8d31a810c75f26a4 [term 1 FOLLOWER]: Refusing update from remote peer d81c4bbc01254e8eaee0c19e0d3d775f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:53:23.434923 23438 consensus_queue.cc:1035] T 6d0667dcb3964557ad1862ddc504f2e0 P d81c4bbc01254e8eaee0c19e0d3d775f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c96cd230adcd495e8d31a810c75f26a4" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 42013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:53:23.435405 23423 consensus_queue.cc:1035] T 6d0667dcb3964557ad1862ddc504f2e0 P d81c4bbc01254e8eaee0c19e0d3d775f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4209b5f7d4d34c82bb07dcb0b78463d3" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 43601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:53:23.450093 23446 mvcc.cc:205] Tried to move back new op lower bound from 7315865614051311616 to 7315865613857005568. Current Snapshot: MvccSnapshot[applied={T|T < 7315865614051311616}]
I20260807 10:53:23.473070 22649 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:53:23.473438 22649 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:53:23.478346 22649 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aa18b7264c0347678a0cc0a30c4fdf03: Sending DeleteTablet for 3 replicas of tablet 6d0667dcb3964557ad1862ddc504f2e0
I20260807 10:53:23.479723 22853 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d0667dcb3964557ad1862ddc504f2e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:53:23 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:55256
I20260807 10:53:23.480224 23077 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d0667dcb3964557ad1862ddc504f2e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:53:23 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:33564
I20260807 10:53:23.481634 23269 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d0667dcb3964557ad1862ddc504f2e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:53:23 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:37200
I20260807 10:53:23.491511 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 22707
I20260807 10:53:23.508899 23462 tablet_replica.cc:330] T 6d0667dcb3964557ad1862ddc504f2e0 P 4209b5f7d4d34c82bb07dcb0b78463d3: stopping tablet replica
I20260807 10:53:23.509694 23459 tablet_replica.cc:330] T 6d0667dcb3964557ad1862ddc504f2e0 P d81c4bbc01254e8eaee0c19e0d3d775f: stopping tablet replica
I20260807 10:53:23.510406 23462 raft_consensus.cc:2229] T 6d0667dcb3964557ad1862ddc504f2e0 P 4209b5f7d4d34c82bb07dcb0b78463d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:53:23.511618 23459 raft_consensus.cc:2229] T 6d0667dcb3964557ad1862ddc504f2e0 P d81c4bbc01254e8eaee0c19e0d3d775f [term 1 LEADER]: Raft consensus shutting down.
I20260807 10:53:23.512316 23462 raft_consensus.cc:2258] T 6d0667dcb3964557ad1862ddc504f2e0 P 4209b5f7d4d34c82bb07dcb0b78463d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:53:23.513413 23459 raft_consensus.cc:2258] T 6d0667dcb3964557ad1862ddc504f2e0 P d81c4bbc01254e8eaee0c19e0d3d775f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:53:23.518499 23462 ts_tablet_manager.cc:1905] T 6d0667dcb3964557ad1862ddc504f2e0 P 4209b5f7d4d34c82bb07dcb0b78463d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:53:23.518827 23459 ts_tablet_manager.cc:1905] T 6d0667dcb3964557ad1862ddc504f2e0 P d81c4bbc01254e8eaee0c19e0d3d775f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:53:23.519680 23456 tablet_replica.cc:330] T 6d0667dcb3964557ad1862ddc504f2e0 P c96cd230adcd495e8d31a810c75f26a4: stopping tablet replica
I20260807 10:53:23.522173 23456 raft_consensus.cc:2229] T 6d0667dcb3964557ad1862ddc504f2e0 P c96cd230adcd495e8d31a810c75f26a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:53:23.527298 23462 ts_tablet_manager.cc:1918] T 6d0667dcb3964557ad1862ddc504f2e0 P 4209b5f7d4d34c82bb07dcb0b78463d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:53:23.527617 23462 log.cc:1196] T 6d0667dcb3964557ad1862ddc504f2e0 P 4209b5f7d4d34c82bb07dcb0b78463d3: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-2/wal/wals/6d0667dcb3964557ad1862ddc504f2e0
I20260807 10:53:23.528366 23462 ts_tablet_manager.cc:1939] T 6d0667dcb3964557ad1862ddc504f2e0 P 4209b5f7d4d34c82bb07dcb0b78463d3: Deleting consensus metadata
I20260807 10:53:23.530676 23459 ts_tablet_manager.cc:1918] T 6d0667dcb3964557ad1862ddc504f2e0 P d81c4bbc01254e8eaee0c19e0d3d775f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:53:23.530704 22629 catalog_manager.cc:4841] TS 4209b5f7d4d34c82bb07dcb0b78463d3 (127.3.73.195:43601): tablet 6d0667dcb3964557ad1862ddc504f2e0 (table test-table [id=eeec37ade68047459ff59a893001fed0]) successfully deleted
I20260807 10:53:23.531032 23459 log.cc:1196] T 6d0667dcb3964557ad1862ddc504f2e0 P d81c4bbc01254e8eaee0c19e0d3d775f: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-1/wal/wals/6d0667dcb3964557ad1862ddc504f2e0
I20260807 10:53:23.531661 23459 ts_tablet_manager.cc:1939] T 6d0667dcb3964557ad1862ddc504f2e0 P d81c4bbc01254e8eaee0c19e0d3d775f: Deleting consensus metadata
I20260807 10:53:23.533332 23456 raft_consensus.cc:2258] T 6d0667dcb3964557ad1862ddc504f2e0 P c96cd230adcd495e8d31a810c75f26a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:53:23.535163 22629 catalog_manager.cc:4841] TS d81c4bbc01254e8eaee0c19e0d3d775f (127.3.73.194:34623): tablet 6d0667dcb3964557ad1862ddc504f2e0 (table test-table [id=eeec37ade68047459ff59a893001fed0]) successfully deleted
I20260807 10:53:23.539501 23456 ts_tablet_manager.cc:1905] T 6d0667dcb3964557ad1862ddc504f2e0 P c96cd230adcd495e8d31a810c75f26a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:53:23.550866 23456 ts_tablet_manager.cc:1918] T 6d0667dcb3964557ad1862ddc504f2e0 P c96cd230adcd495e8d31a810c75f26a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:53:23.559681 23408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:23.493483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55262 (local address 127.3.73.193:42013)
0807 10:53:23.493690 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:23.493698 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:23.511163 (+ 17465us) server_negotiation.cc:407] Connection header received
0807 10:53:23.513701 (+ 2538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:23.513707 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:23.513843 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:23.513925 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:23.519969 (+ 6044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:23.521132 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:23.532800 (+ 11668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:23.533550 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:23.533692 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:23.551341 (+ 17649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:23.551360 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:23.551384 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:23.551438 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:23.555556 (+ 4118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:23.556816 (+ 1260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:23.556827 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:23.556830 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:23.557021 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:23.557567 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:23.557572 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:23.557575 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:23.558063 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:23.558365 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:23.559002 (+ 637us) server_negotiation.cc:299] Negotiation successful
0807 10:53:23.559276 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 10:53:23.561612 22947 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:53:23.565249 23456 log.cc:1196] T 6d0667dcb3964557ad1862ddc504f2e0 P c96cd230adcd495e8d31a810c75f26a4: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786099767749326-3367-0/minicluster-data/ts-0/wal/wals/6d0667dcb3964557ad1862ddc504f2e0
I20260807 10:53:23.730419 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 22707
I20260807 10:53:23.730438 23456 ts_tablet_manager.cc:1939] T 6d0667dcb3964557ad1862ddc504f2e0 P c96cd230adcd495e8d31a810c75f26a4: Deleting consensus metadata
W20260807 10:53:23.769683 22627 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.73.193:42013 (error 108)
I20260807 10:53:23.770192 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 22966
I20260807 10:53:23.790988 23406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:23.771226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33570 (local address 127.3.73.194:34623)
0807 10:53:23.771442 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:23.771448 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:23.772040 (+ 592us) server_negotiation.cc:407] Connection header received
0807 10:53:23.772312 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:23.772318 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:23.772397 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:23.772492 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:23.774716 (+ 2224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:23.775921 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:23.777952 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:23.778662 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:23.778775 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:23.782677 (+ 3902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:23.782693 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:23.782713 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:23.782762 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:23.787042 (+ 4280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:23.788013 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:23.788024 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:23.788027 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:23.788266 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:23.788830 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:23.788835 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:23.788839 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:23.789370 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:23.789719 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:23.790343 (+ 624us) server_negotiation.cc:299] Negotiation successful
0807 10:53:23.790588 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 10:53:23.793037 23170 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:53:23.793699 23471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:23.771956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:42013 (local address 127.0.0.1:55272)
0807 10:53:23.793317 (+ 21361us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:23.793488 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.193:42013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21086,"threads_started":1}
W20260807 10:53:23.794306 22627 catalog_manager.cc:4584] TS c96cd230adcd495e8d31a810c75f26a4 (127.3.73.193:42013): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6d0667dcb3964557ad1862ddc504f2e0: Network error: Client connection negotiation failed: client connection to 127.3.73.193:42013: connect: Connection refused (error 111)
I20260807 10:53:23.848364 23471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:23.847734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:42013 (local address 127.0.0.1:55278)
0807 10:53:23.848004 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:23.848143 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.193:42013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260807 10:53:23.849751 23471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:23.849275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:42013 (local address 127.0.0.1:55288)
0807 10:53:23.849454 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:23.849566 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.193:42013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 10:53:23.910682 23471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:23.910069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:42013 (local address 127.0.0.1:55290)
0807 10:53:23.910404 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:23.910512 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.193:42013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166}
I20260807 10:53:23.911927 23471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:23.911459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:42013 (local address 127.0.0.1:55306)
0807 10:53:23.911659 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:23.911745 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.193:42013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260807 10:53:23.922639 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 22966
I20260807 10:53:23.954276 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 23185
I20260807 10:53:23.978660 23474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:23.955294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37206 (local address 127.3.73.195:43601)
0807 10:53:23.956218 (+ 924us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:23.956228 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:23.956257 (+ 29us) server_negotiation.cc:407] Connection header received
0807 10:53:23.956336 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:23.956342 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:23.956429 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:23.958477 (+ 2048us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:23.958680 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:23.959704 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:23.961942 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:23.962619 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:23.963504 (+ 885us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:23.969759 (+ 6255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:23.969777 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:23.969793 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:23.969837 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:23.973170 (+ 3333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:23.974411 (+ 1241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:23.974421 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:23.974423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:23.974667 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:23.976131 (+ 1464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:23.976137 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:23.976142 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:23.977071 (+ 929us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:23.977683 (+ 612us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:23.977905 (+ 222us) server_negotiation.cc:299] Negotiation successful
0807 10:53:23.978371 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":741,"thread_start_us":583,"threads_started":1}
I20260807 10:53:23.979358 23365 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:53:23.992678 23471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:23.992018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:42013 (local address 127.0.0.1:55318)
0807 10:53:23.992298 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:23.992473 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.193:42013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260807 10:53:23.994429 23471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:23.993539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:42013 (local address 127.0.0.1:55324)
0807 10:53:23.993876 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:23.994120 (+ 244us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.193:42013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157}
I20260807 10:53:24.077714 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 23185
I20260807 10:53:24.109110 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 22598
I20260807 10:53:24.110613 22696 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:53:24.231433 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 22598
I20260807 10:53:24.265630 3367 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 10:53:24.612665 3367 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 10:53:24.613034 3367 mini_ranger.cc:66] Stopping Ranger...
I20260807 10:53:24.970501 3367 mini_ranger.cc:68] Stopped Ranger
2026-08-07 10:53:24.970 UTC [21102] LOG: received smart shutdown request
2026-08-07 10:53:24.975 UTC [21102] LOG: background worker "logical replication launcher" (PID 21113) exited with exit code 1
2026-08-07 10:53:24.975 UTC [21106] LOG: shutting down
2026-08-07 10:53:25.189 UTC [21102] LOG: database system is shut down
2026-08-07T10:53:25Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (61482 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:53:25 dist-test-slave-x5fp krb5kdc[23487](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:53:25 dist-test-slave-x5fp krb5kdc[23487](info): set up 2 sockets
Aug 07 10:53:25 dist-test-slave-x5fp krb5kdc[23487](info): commencing operation
krb5kdc: starting...
W20260807 10:53:26.706926 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.332s user 0.003s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:53:26 dist-test-slave-x5fp krb5kdc[23487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:53:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:53:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.73.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
I20260807 10:53:27.056670 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:45425
--webserver_interface=127.3.73.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:35639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:45425,127.3.73.253:43503,127.3.73.252:36205
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:53:27.447331 23503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:53:27.447676 23503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:53:27.447870 23503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:53:27.455476 23503 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:53:27.455615 23503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:53:27.455703 23503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:53:27.455796 23503 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:53:27.455865 23503 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:53:27.468101 23503 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:45425,127.3.73.253:43503,127.3.73.252:36205
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:45425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:53:27.470152 23503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:53:27.472285 23503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:53:27.486953 23514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:27.486989 23512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:27.487021 23516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:53:27.487948 23503 server_base.cc:1034] running on GCE node
I20260807 10:53:27.489001 23503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:53:27.490586 23503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:53:27.491868 23503 hybrid_clock.cc:648] HybridClock initialized: now 1786100007491786 us; error 75 us; skew 500 ppm
I20260807 10:53:27.492281 23503 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:53:27 dist-test-slave-x5fp krb5kdc[23487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100007, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:53:27.496275 23503 init.cc:377] Logged in from keytab as kudu/127.3.73.254@KRBTEST.COM (short username kudu)
I20260807 10:53:27.498451 23503 webserver.cc:460] Webserver started at http://127.3.73.254:34175/ using document root <none> and password file <none>
I20260807 10:53:27.499601 23503 fs_manager.cc:359] Metadata directory not provided
I20260807 10:53:27.499740 23503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:53:27.500149 23503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:53:27.504052 23503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "e0800f1aa8ca4e5289677eb5c6cbe151"
format_stamp: "Formatted at 2026-08-07 10:53:27 on dist-test-slave-x5fp"
server_key: "eb45d579b57d0f1fcbc19adefcbadc87"
server_key_iv: "20ffc32628cbe209b4564f3095a073a3"
server_key_version: "encryptionkey@0"
I20260807 10:53:27.505081 23503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "e0800f1aa8ca4e5289677eb5c6cbe151"
format_stamp: "Formatted at 2026-08-07 10:53:27 on dist-test-slave-x5fp"
server_key: "eb45d579b57d0f1fcbc19adefcbadc87"
server_key_iv: "20ffc32628cbe209b4564f3095a073a3"
server_key_version: "encryptionkey@0"
I20260807 10:53:27.512336 23503 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.005s
I20260807 10:53:27.517019 23528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:27.518682 23503 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 10:53:27.518972 23503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "e0800f1aa8ca4e5289677eb5c6cbe151"
format_stamp: "Formatted at 2026-08-07 10:53:27 on dist-test-slave-x5fp"
server_key: "eb45d579b57d0f1fcbc19adefcbadc87"
server_key_iv: "20ffc32628cbe209b4564f3095a073a3"
server_key_version: "encryptionkey@0"
I20260807 10:53:27.519218 23503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:53:27.551463 23503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:53:27.556567 23503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:53:27.556936 23503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:53:27.581112 23503 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:45425
I20260807 10:53:27.581243 23607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:45425 every 8 connection(s)
I20260807 10:53:27.582695 23503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:53:27.587116 23608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:53:27.592500 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 23503
I20260807 10:53:27.592779 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:53:27.593665 3367 external_mini_cluster.cc:1428] Setting key c16fff539f572535e1ebb0f4d690f6ad
I20260807 10:53:27.595614 23608 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45425 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 10:53:27 dist-test-slave-x5fp krb5kdc[23487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100007, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:53:27.624125 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:27.599757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:45425 (local address 127.0.0.1:60868)
0807 10:53:27.600756 (+ 999us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:27.600806 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 10:53:27.603039 (+ 2233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:53:27.604059 (+ 1020us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:53:27.604064 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:53:27.604830 (+ 766us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:53:27.605191 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:27.605209 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:27.607350 (+ 2141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:27.607358 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:27.609388 (+ 2030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:27.609414 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:27.610502 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:27.610512 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:27.610747 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:27.611524 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:53:27.611566 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:53:27.614566 (+ 3000us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:53:27.619393 (+ 4827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:27.619405 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:27.619416 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:27.619880 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:27.620448 (+ 568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:27.620452 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:27.620454 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:27.620628 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:27.621764 (+ 1136us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:53:27.621770 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:53:27.622004 (+ 234us) client_negotiation.cc:769] Sending connection context
0807 10:53:27.622234 (+ 230us) client_negotiation.cc:240] Negotiation successful
0807 10:53:27.622784 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":684,"thread_start_us":421,"threads_started":1}
I20260807 10:53:27.624846 23612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:27.601292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60868 (local address 127.3.73.254:45425)
0807 10:53:27.602034 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:27.602045 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:27.602099 (+ 54us) server_negotiation.cc:407] Connection header received
0807 10:53:27.603463 (+ 1364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:27.603474 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:27.603790 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:27.604031 (+ 241us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:27.605408 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:27.607176 (+ 1768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:27.609613 (+ 2437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:27.610212 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:27.610359 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:27.614872 (+ 4513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:27.614891 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:27.614915 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:27.614975 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:27.619206 (+ 4231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:27.620064 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:27.620078 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:27.620083 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:27.620305 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:27.620781 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:27.620789 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:27.620794 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:27.621603 (+ 809us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:27.621826 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:27.622463 (+ 637us) server_negotiation.cc:299] Negotiation successful
0807 10:53:27.623003 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":174,"server-negotiator.queue_time_us":509,"thread_start_us":326,"threads_started":1}
WARNING: no policy specified for kudu/127.3.73.253@KRBTEST.COM; defaulting to no policy
I20260807 10:53:27.628224 23608 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.73.253" port: 43503 } has no permanent_uuid. Determining permanent_uuid...
I20260807 10:53:27.629339 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:27.628949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:43503 (local address 127.0.0.1:55074)
0807 10:53:27.629117 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:27.629225 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.253:43503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260807 10:53:27.629630 23530 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.73.253:43503: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.253@KRBTEST.COM" created.
I20260807 10:53:27.631313 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:27.630950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:43503 (local address 127.0.0.1:55088)
0807 10:53:27.631097 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:27.631186 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.253:43503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 10:53:27.631543 23608 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.253:43503: Network error: Client connection negotiation failed: client connection to 127.3.73.253:43503: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.253.keytab.
Entry for principal kudu/127.3.73.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.253.keytab.
I20260807 10:53:27.669653 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.253:43503
--webserver_interface=127.3.73.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.253.keytab
--principal=kudu/127.3.73.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:35639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:45425,127.3.73.253:43503,127.3.73.252:36205
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 10:53:27.680985 23608 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.253" port: 43503 } attempt: 1
I20260807 10:53:27.682314 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:27.681796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:43503 (local address 127.0.0.1:55102)
0807 10:53:27.682051 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:27.682150 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.253:43503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260807 10:53:27.683459 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:27.683075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:43503 (local address 127.0.0.1:55116)
0807 10:53:27.683238 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:27.683317 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.253:43503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 10:53:27.683646 23608 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.253:43503: Network error: Client connection negotiation failed: client connection to 127.3.73.253:43503: connect: Connection refused (error 111)
I20260807 10:53:27.751924 23608 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.253" port: 43503 } attempt: 2
I20260807 10:53:27.753384 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:27.752852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:43503 (local address 127.0.0.1:55124)
0807 10:53:27.753104 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:27.753202 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.253:43503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260807 10:53:27.754489 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:27.754100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:43503 (local address 127.0.0.1:55140)
0807 10:53:27.754256 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:27.754334 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.253:43503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 10:53:27.754669 23608 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.253:43503: Network error: Client connection negotiation failed: client connection to 127.3.73.253:43503: connect: Connection refused (error 111)
I20260807 10:53:27.845989 23608 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.253" port: 43503 } attempt: 3
I20260807 10:53:27.847401 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:27.846915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:43503 (local address 127.0.0.1:55142)
0807 10:53:27.847128 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:27.847229 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.253:43503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260807 10:53:27.848483 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:27.848091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:43503 (local address 127.0.0.1:55154)
0807 10:53:27.848241 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:27.848344 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.253:43503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 10:53:27.848665 23608 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.253:43503: Network error: Client connection negotiation failed: client connection to 127.3.73.253:43503: connect: Connection refused (error 111)
I20260807 10:53:27.992035 23608 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.253" port: 43503 } attempt: 4
I20260807 10:53:27.993526 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:27.992891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:43503 (local address 127.0.0.1:55158)
0807 10:53:27.993210 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:27.993378 (+ 168us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.253:43503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181}
I20260807 10:53:27.994768 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:27.994332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:43503 (local address 127.0.0.1:55172)
0807 10:53:27.994505 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:27.994617 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.253:43503: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260807 10:53:27.994973 23608 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.253:43503: Network error: Client connection negotiation failed: client connection to 127.3.73.253:43503: connect: Connection refused (error 111)
W20260807 10:53:28.049062 23617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:53:28.049471 23617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:53:28.049642 23617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:53:28.057638 23617 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:53:28.057813 23617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:53:28.057924 23617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:53:28.058039 23617 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:53:28.058120 23617 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:53:28.069491 23617 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:45425,127.3.73.253:43503,127.3.73.252:36205
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.253.keytab
--principal=kudu/127.3.73.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.253:43503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.73.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:53:28.071116 23617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:53:28.072762 23617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:53:28.087831 23633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:28.087863 23628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:28.087945 23623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:53:28.090691 23617 server_base.cc:1034] running on GCE node
I20260807 10:53:28.091730 23617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:53:28.093603 23617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:53:28.095021 23617 hybrid_clock.cc:648] HybridClock initialized: now 1786100008094947 us; error 69 us; skew 500 ppm
I20260807 10:53:28.095465 23617 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:53:28 dist-test-slave-x5fp krb5kdc[23487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100008, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:53:28.099097 23617 init.cc:377] Logged in from keytab as kudu/127.3.73.253@KRBTEST.COM (short username kudu)
I20260807 10:53:28.100983 23617 webserver.cc:460] Webserver started at http://127.3.73.253:34783/ using document root <none> and password file <none>
I20260807 10:53:28.101820 23617 fs_manager.cc:359] Metadata directory not provided
I20260807 10:53:28.101997 23617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:53:28.102346 23617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:53:28.105856 23617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-1/data/instance:
uuid: "73399a33ca8a491983e03ba207b45b44"
format_stamp: "Formatted at 2026-08-07 10:53:28 on dist-test-slave-x5fp"
server_key: "1c51b444d213003b32142b40c7821236"
server_key_iv: "d479acc0e5be627de9b3f9a2265a2a75"
server_key_version: "encryptionkey@0"
I20260807 10:53:28.106855 23617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-1/wal/instance:
uuid: "73399a33ca8a491983e03ba207b45b44"
format_stamp: "Formatted at 2026-08-07 10:53:28 on dist-test-slave-x5fp"
server_key: "1c51b444d213003b32142b40c7821236"
server_key_iv: "d479acc0e5be627de9b3f9a2265a2a75"
server_key_version: "encryptionkey@0"
I20260807 10:53:28.114235 23617 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260807 10:53:28.121040 23642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:28.122912 23617 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 10:53:28.123169 23617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-1/wal
uuid: "73399a33ca8a491983e03ba207b45b44"
format_stamp: "Formatted at 2026-08-07 10:53:28 on dist-test-slave-x5fp"
server_key: "1c51b444d213003b32142b40c7821236"
server_key_iv: "d479acc0e5be627de9b3f9a2265a2a75"
server_key_version: "encryptionkey@0"
I20260807 10:53:28.123438 23617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:53:28.145401 23617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:53:28.150887 23617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:53:28.151255 23617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:53:28.173977 23617 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.253:43503
I20260807 10:53:28.174031 23715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.253:43503 every 8 connection(s)
I20260807 10:53:28.175570 23617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-1/data/info.pb
I20260807 10:53:28.180900 23716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:53:28.185397 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 23617
I20260807 10:53:28.185626 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-1/wal/instance
I20260807 10:53:28.186225 3367 external_mini_cluster.cc:1428] Setting key 367b9e6ef8392a11183e016aeda8381c
I20260807 10:53:28.189791 23716 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45425 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 10:53:28 dist-test-slave-x5fp krb5kdc[23487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100008, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.253@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:53:28.214197 23720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.193676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60872 (local address 127.3.73.254:45425)
0807 10:53:28.194358 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:28.194366 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:28.195421 (+ 1055us) server_negotiation.cc:407] Connection header received
0807 10:53:28.197597 (+ 2176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:28.197606 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:28.197711 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:28.197851 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:28.199494 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.200248 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.201994 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.202434 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.202581 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:28.206775 (+ 4194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:28.206781 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:28.206785 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:28.206841 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:28.209973 (+ 3132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:28.211113 (+ 1140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:28.211119 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:28.211123 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:28.211282 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:28.211789 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:28.211794 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:28.211797 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:28.212246 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:28.212494 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:28.213616 (+ 1122us) server_negotiation.cc:299] Negotiation successful
0807 10:53:28.213880 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":371,"threads_started":1}
I20260807 10:53:28.215801 23721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.194294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:45425 (local address 127.0.0.1:60872)
0807 10:53:28.195225 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:28.195282 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 10:53:28.197294 (+ 2012us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:53:28.198087 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:53:28.198096 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:53:28.198656 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:53:28.199282 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:28.199301 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.200446 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.200450 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:28.201814 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:28.201839 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.202704 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.202711 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:28.202950 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:28.203690 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:53:28.203716 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:53:28.206445 (+ 2729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:53:28.210187 (+ 3742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:28.210201 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:28.210218 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:28.210751 (+ 533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:28.211414 (+ 663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:28.211421 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:28.211429 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:28.211643 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:28.212443 (+ 800us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:53:28.212453 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:53:28.213080 (+ 627us) client_negotiation.cc:769] Sending connection context
0807 10:53:28.213467 (+ 387us) client_negotiation.cc:240] Negotiation successful
0807 10:53:28.214143 (+ 676us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":418,"threads_started":1}
WARNING: no policy specified for kudu/127.3.73.252@KRBTEST.COM; defaulting to no policy
I20260807 10:53:28.218001 23716 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.73.253" port: 43503 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.252@KRBTEST.COM" created.
Aug 07 10:53:28 dist-test-slave-x5fp krb5kdc[23487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100008, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.253@KRBTEST.COM for kudu/127.3.73.253@KRBTEST.COM
I20260807 10:53:28.238269 23721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.218750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:43503 (local address 127.0.0.1:55176)
0807 10:53:28.219007 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:28.219029 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 10:53:28.219134 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:53:28.220300 (+ 1166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:53:28.220304 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:53:28.220974 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:53:28.221178 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:28.221193 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.222850 (+ 1657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.222859 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:28.224409 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:28.224432 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.225719 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.225729 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:28.225996 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:28.226770 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:53:28.226798 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:53:28.229980 (+ 3182us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:53:28.235039 (+ 5059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:28.235048 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:28.235054 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:28.235691 (+ 637us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:28.236203 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:28.236210 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:28.236214 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:28.236443 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:28.237428 (+ 985us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:53:28.237432 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:53:28.237594 (+ 162us) client_negotiation.cc:769] Sending connection context
0807 10:53:28.237719 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 10:53:28.237999 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100}
I20260807 10:53:28.238785 23724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.219103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55176 (local address 127.3.73.253:43503)
0807 10:53:28.219793 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:28.219800 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:28.219830 (+ 30us) server_negotiation.cc:407] Connection header received
0807 10:53:28.219879 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:28.219885 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:28.220081 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:28.220317 (+ 236us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:28.221393 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.222674 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.224667 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.225506 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.225639 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:28.230258 (+ 4619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:28.230277 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:28.230302 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:28.230356 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:28.234807 (+ 4451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:28.235906 (+ 1099us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:28.235914 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:28.235916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:28.236051 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:28.236602 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:28.236610 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:28.236616 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:28.237257 (+ 641us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:28.237498 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:28.237919 (+ 421us) server_negotiation.cc:299] Negotiation successful
0807 10:53:28.238231 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":398,"threads_started":1}
I20260807 10:53:28.241320 23716 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.73.252" port: 36205 } has no permanent_uuid. Determining permanent_uuid...
I20260807 10:53:28.242537 23721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.242054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:36205 (local address 127.0.0.1:58426)
0807 10:53:28.242262 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:28.242375 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:36205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260807 10:53:28.242784 23643 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.73.252:36205: connect: Connection refused (error 111)
I20260807 10:53:28.244454 23721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.244099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:36205 (local address 127.0.0.1:58436)
0807 10:53:28.244252 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:28.244343 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:36205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 10:53:28.244714 23716 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.252:36205: Network error: Client connection negotiation failed: client connection to 127.3.73.252:36205: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.252.keytab.
Entry for principal kudu/127.3.73.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.252.keytab.
I20260807 10:53:28.265138 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.252:36205
--webserver_interface=127.3.73.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.252.keytab
--principal=kudu/127.3.73.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:35639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:45425,127.3.73.253:43503,127.3.73.252:36205
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 10:53:28.294257 23608 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.253" port: 43503 } attempt: 5
I20260807 10:53:28.294175 23716 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.252" port: 36205 } attempt: 1
I20260807 10:53:28.295746 23721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.295083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:36205 (local address 127.0.0.1:58448)
0807 10:53:28.295331 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:28.295483 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:36205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260807 10:53:28.297381 23721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.296779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:36205 (local address 127.0.0.1:58450)
0807 10:53:28.297065 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:28.297186 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:36205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
W20260807 10:53:28.297676 23716 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.252:36205: Network error: Client connection negotiation failed: client connection to 127.3.73.252:36205: connect: Connection refused (error 111)
Aug 07 10:53:28 dist-test-slave-x5fp krb5kdc[23487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100007, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for kudu/127.3.73.253@KRBTEST.COM
I20260807 10:53:28.310770 23724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.295372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55186 (local address 127.3.73.253:43503)
0807 10:53:28.295580 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:28.295589 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:28.295619 (+ 30us) server_negotiation.cc:407] Connection header received
0807 10:53:28.295794 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:28.295802 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:28.295932 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:28.296063 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:28.297281 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.298430 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.299830 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.300377 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.300537 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:28.304141 (+ 3604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:28.304147 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:28.304151 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:28.304189 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:28.307197 (+ 3008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:28.308173 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:28.308180 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:28.308184 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:28.308351 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:28.308909 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:28.308917 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:28.308920 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:28.309314 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:28.309505 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:28.309994 (+ 489us) server_negotiation.cc:299] Negotiation successful
0807 10:53:28.310401 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 10:53:28.310773 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.295100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:43503 (local address 127.0.0.1:55186)
0807 10:53:28.295446 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:28.295467 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 10:53:28.295613 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:53:28.296128 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:53:28.296135 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:53:28.296698 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:53:28.297075 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:28.297093 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.298591 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.298597 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:28.299659 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:28.299675 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.300554 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.300561 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:28.300813 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:28.301555 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:53:28.301576 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:53:28.303879 (+ 2303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:53:28.307521 (+ 3642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:28.307528 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:28.307533 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:28.308020 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:28.308496 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:28.308501 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:28.308506 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:28.308671 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:28.309460 (+ 789us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:53:28.309465 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:53:28.309644 (+ 179us) client_negotiation.cc:769] Sending connection context
0807 10:53:28.309857 (+ 213us) client_negotiation.cc:240] Negotiation successful
0807 10:53:28.310329 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221}
I20260807 10:53:28.312286 23608 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.73.252" port: 36205 } has no permanent_uuid. Determining permanent_uuid...
I20260807 10:53:28.313558 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.313086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:36205 (local address 127.0.0.1:58464)
0807 10:53:28.313281 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:28.313386 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:36205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 10:53:28.314713 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.314339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:36205 (local address 127.0.0.1:58470)
0807 10:53:28.314509 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:28.314591 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:36205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260807 10:53:28.315063 23608 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.252:36205: Network error: Client connection negotiation failed: client connection to 127.3.73.252:36205: connect: Connection refused (error 111)
I20260807 10:53:28.365979 23716 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.252" port: 36205 } attempt: 2
I20260807 10:53:28.366398 23608 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.252" port: 36205 } attempt: 1
I20260807 10:53:28.367434 23721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.366873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:36205 (local address 127.0.0.1:58476)
0807 10:53:28.367134 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:28.367249 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:36205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260807 10:53:28.367615 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.367083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:36205 (local address 127.0.0.1:58492)
0807 10:53:28.367303 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:28.367412 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:36205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260807 10:53:28.368757 23721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.368300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:36205 (local address 127.0.0.1:58506)
0807 10:53:28.368480 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:28.368573 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:36205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260807 10:53:28.368829 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.368394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:36205 (local address 127.0.0.1:58514)
0807 10:53:28.368552 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:28.368650 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:36205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 10:53:28.368999 23716 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.252:36205: Network error: Client connection negotiation failed: client connection to 127.3.73.252:36205: connect: Connection refused (error 111)
W20260807 10:53:28.369094 23608 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.252:36205: Network error: Client connection negotiation failed: client connection to 127.3.73.252:36205: connect: Connection refused (error 111)
I20260807 10:53:28.437373 23608 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.252" port: 36205 } attempt: 2
I20260807 10:53:28.439011 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.438299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:36205 (local address 127.0.0.1:58520)
0807 10:53:28.438608 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:28.438750 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:36205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260807 10:53:28.440344 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.439907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:36205 (local address 127.0.0.1:58536)
0807 10:53:28.440088 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:28.440181 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:36205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260807 10:53:28.440668 23608 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.252:36205: Network error: Client connection negotiation failed: client connection to 127.3.73.252:36205: connect: Connection refused (error 111)
I20260807 10:53:28.460278 23716 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.252" port: 36205 } attempt: 3
I20260807 10:53:28.461736 23721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.461225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:36205 (local address 127.0.0.1:58542)
0807 10:53:28.461457 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:28.461563 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:36205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260807 10:53:28.463028 23721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.462537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:36205 (local address 127.0.0.1:58546)
0807 10:53:28.462807 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:28.462900 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:36205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153}
W20260807 10:53:28.463241 23716 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.252:36205: Network error: Client connection negotiation failed: client connection to 127.3.73.252:36205: connect: Connection refused (error 111)
I20260807 10:53:28.547093 23608 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.252" port: 36205 } attempt: 3
I20260807 10:53:28.548641 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.547929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:36205 (local address 127.0.0.1:58550)
0807 10:53:28.548225 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:28.548382 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:36205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260807 10:53:28.550319 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.549727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:36205 (local address 127.0.0.1:58554)
0807 10:53:28.549960 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:28.550122 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:36205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260807 10:53:28.550642 23608 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.252:36205: Network error: Client connection negotiation failed: client connection to 127.3.73.252:36205: connect: Connection refused (error 111)
I20260807 10:53:28.606483 23716 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.252" port: 36205 } attempt: 4
I20260807 10:53:28.607918 23721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.607331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:36205 (local address 127.0.0.1:58564)
0807 10:53:28.607610 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:28.607706 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:36205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260807 10:53:28.609213 23721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.608840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:36205 (local address 127.0.0.1:58566)
0807 10:53:28.609027 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:28.609101 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:36205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 10:53:28.609462 23716 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.252:36205: Network error: Client connection negotiation failed: client connection to 127.3.73.252:36205: connect: Connection refused (error 111)
W20260807 10:53:28.660646 23734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:53:28.661113 23734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:53:28.661309 23734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:53:28.668960 23734 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:53:28.669078 23734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:53:28.669190 23734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:53:28.669324 23734 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:53:28.669401 23734 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:53:28.679839 23734 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:45425,127.3.73.253:43503,127.3.73.252:36205
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.252.keytab
--principal=kudu/127.3.73.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.252:36205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.73.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:53:28.681443 23734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:53:28.682861 23734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:53:28.695568 23741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:28.695585 23740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:28.695585 23744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:53:28.696405 23734 server_base.cc:1034] running on GCE node
I20260807 10:53:28.697299 23734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:53:28.698896 23734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:53:28.700146 23734 hybrid_clock.cc:648] HybridClock initialized: now 1786100008700086 us; error 61 us; skew 500 ppm
I20260807 10:53:28.700500 23734 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:53:28 dist-test-slave-x5fp krb5kdc[23487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100008, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:53:28.704075 23734 init.cc:377] Logged in from keytab as kudu/127.3.73.252@KRBTEST.COM (short username kudu)
I20260807 10:53:28.705780 23734 webserver.cc:460] Webserver started at http://127.3.73.252:37363/ using document root <none> and password file <none>
I20260807 10:53:28.706575 23734 fs_manager.cc:359] Metadata directory not provided
I20260807 10:53:28.706722 23734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:53:28.707077 23734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:53:28.710461 23734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-2/data/instance:
uuid: "a2e3ddb225d7433d9d0c65a0783f1943"
format_stamp: "Formatted at 2026-08-07 10:53:28 on dist-test-slave-x5fp"
server_key: "8a824bd6c5f4352f18735806ec93931e"
server_key_iv: "8e0129bc8976cd3c2db621fcf2660f98"
server_key_version: "encryptionkey@0"
I20260807 10:53:28.711465 23734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-2/wal/instance:
uuid: "a2e3ddb225d7433d9d0c65a0783f1943"
format_stamp: "Formatted at 2026-08-07 10:53:28 on dist-test-slave-x5fp"
server_key: "8a824bd6c5f4352f18735806ec93931e"
server_key_iv: "8e0129bc8976cd3c2db621fcf2660f98"
server_key_version: "encryptionkey@0"
I20260807 10:53:28.717654 23734 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 10:53:28.721778 23757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:28.723608 23734 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 10:53:28.723852 23734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-2/wal
uuid: "a2e3ddb225d7433d9d0c65a0783f1943"
format_stamp: "Formatted at 2026-08-07 10:53:28 on dist-test-slave-x5fp"
server_key: "8a824bd6c5f4352f18735806ec93931e"
server_key_iv: "8e0129bc8976cd3c2db621fcf2660f98"
server_key_version: "encryptionkey@0"
I20260807 10:53:28.724124 23734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:53:28.727928 23608 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.252" port: 36205 } attempt: 4
I20260807 10:53:28.729185 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.728709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:36205 (local address 127.0.0.1:58572)
0807 10:53:28.728930 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:28.729028 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:36205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260807 10:53:28.730501 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.730005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:36205 (local address 127.0.0.1:58576)
0807 10:53:28.730213 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:28.730319 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:36205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260807 10:53:28.730721 23608 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.252:36205: Network error: Client connection negotiation failed: client connection to 127.3.73.252:36205: connect: Connection refused (error 111)
I20260807 10:53:28.743012 23734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:53:28.748055 23734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:53:28.748323 23734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:53:28.769651 23734 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.252:36205
I20260807 10:53:28.769740 23833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.252:36205 every 8 connection(s)
I20260807 10:53:28.771054 23734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-2/data/info.pb
I20260807 10:53:28.775207 23834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:53:28.780963 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 23734
I20260807 10:53:28.781162 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786099767749326-3367-0/minicluster-data/master-2/wal/instance
I20260807 10:53:28.781623 3367 external_mini_cluster.cc:1428] Setting key a0a861fcefde1f053259722cc6b9b934
I20260807 10:53:28.783118 23834 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45425 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 10:53:28 dist-test-slave-x5fp krb5kdc[23487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100008, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.252@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
Aug 07 10:53:28 dist-test-slave-x5fp krb5kdc[23487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.252@KRBTEST.COM
I20260807 10:53:28.806406 23841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.786189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60880 (local address 127.3.73.254:45425)
0807 10:53:28.787521 (+ 1332us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:28.787529 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:28.787552 (+ 23us) server_negotiation.cc:407] Connection header received
0807 10:53:28.787617 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:28.787622 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:28.787710 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:28.787832 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:28.789694 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.790966 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.793225 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.793882 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.794022 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:28.798895 (+ 4873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:28.798906 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:28.798912 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:28.798970 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:28.802441 (+ 3471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:28.803309 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:28.803315 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:28.803318 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:28.803525 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:28.804032 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:28.804038 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:28.804043 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:28.804411 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:28.804544 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:28.805459 (+ 915us) server_negotiation.cc:299] Negotiation successful
0807 10:53:28.805736 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1204,"thread_start_us":1028,"threads_started":1}
I20260807 10:53:28.807375 23839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.786081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:45425 (local address 127.0.0.1:60880)
0807 10:53:28.786983 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:28.787051 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0807 10:53:28.787359 (+ 308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:53:28.787946 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:53:28.787955 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:53:28.788812 (+ 857us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:53:28.789456 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:28.789484 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.791146 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.791150 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:28.793067 (+ 1917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:28.793086 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.794125 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.794130 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:28.794340 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:28.795201 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:53:28.795232 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:53:28.798565 (+ 3333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:53:28.802644 (+ 4079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:28.802658 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:28.802674 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:28.803145 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:28.803654 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:28.803658 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:28.803661 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:28.803876 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:28.804545 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:53:28.804553 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:53:28.805019 (+ 466us) client_negotiation.cc:769] Sending connection context
0807 10:53:28.805306 (+ 287us) client_negotiation.cc:240] Negotiation successful
0807 10:53:28.805888 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"mutex_wait_us":93,"thread_start_us":337,"threads_started":1}
I20260807 10:53:28.807399 23837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.783580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58590 (local address 127.3.73.252:36205)
0807 10:53:28.784460 (+ 880us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:28.784476 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:28.784519 (+ 43us) server_negotiation.cc:407] Connection header received
0807 10:53:28.786936 (+ 2417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:28.786962 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:28.787407 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:28.787652 (+ 245us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:28.789698 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.791398 (+ 1700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.793363 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.794232 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.794343 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:28.799484 (+ 5141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:28.799492 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:28.799507 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:28.799537 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:28.802616 (+ 3079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:28.803514 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:28.803521 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:28.803523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:28.803746 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:28.804261 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:28.804265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:28.804267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:28.805055 (+ 788us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:28.805373 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:28.805769 (+ 396us) server_negotiation.cc:299] Negotiation successful
0807 10:53:28.806010 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":660,"spinlock_wait_cycles":14720,"thread_start_us":431,"threads_started":1}
I20260807 10:53:28.810482 23834 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.73.253" port: 43503 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 10:53:28 dist-test-slave-x5fp krb5kdc[23487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100008, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.252@KRBTEST.COM for kudu/127.3.73.253@KRBTEST.COM
Aug 07 10:53:28 dist-test-slave-x5fp krb5kdc[23487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:53:28.826545 23839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.811351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:43503 (local address 127.0.0.1:55190)
0807 10:53:28.811559 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:28.811582 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 10:53:28.811709 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:53:28.812604 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:53:28.812608 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:53:28.813049 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:53:28.813207 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:28.813215 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.814336 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.814342 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:28.815347 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:28.815359 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.816240 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.816248 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:28.816483 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:28.817275 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:53:28.817295 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:53:28.819486 (+ 2191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:53:28.823423 (+ 3937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:28.823432 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:28.823438 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:28.824155 (+ 717us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:28.824601 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:28.824607 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:28.824612 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:28.824878 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:28.825537 (+ 659us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:53:28.825543 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:53:28.825767 (+ 224us) client_negotiation.cc:769] Sending connection context
0807 10:53:28.825956 (+ 189us) client_negotiation.cc:240] Negotiation successful
0807 10:53:28.826249 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 10:53:28.826992 23724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.811929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55190 (local address 127.3.73.253:43503)
0807 10:53:28.812071 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:28.812075 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:28.812089 (+ 14us) server_negotiation.cc:407] Connection header received
0807 10:53:28.812171 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:28.812176 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:28.812260 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:28.812378 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:28.813373 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.814176 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.815512 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.816077 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.816177 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:28.819738 (+ 3561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:28.819749 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:28.819755 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:28.819818 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:28.823239 (+ 3421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:28.824299 (+ 1060us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:28.824303 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:28.824306 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:28.824463 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:28.825076 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:28.825082 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:28.825088 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:28.825392 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:28.825520 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:28.826099 (+ 579us) server_negotiation.cc:299] Negotiation successful
0807 10:53:28.826343 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 10:53:28.828519 23834 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.73.252" port: 36205 } has no permanent_uuid. Determining permanent_uuid...
I20260807 10:53:28.829416 23841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.811613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60882 (local address 127.3.73.254:45425)
0807 10:53:28.811822 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:28.811828 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:28.811850 (+ 22us) server_negotiation.cc:407] Connection header received
0807 10:53:28.811928 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:28.811935 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:28.812003 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:28.812126 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:28.813632 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.814386 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.815862 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.816449 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.816579 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:28.822084 (+ 5505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:28.822091 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:28.822095 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:28.822138 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:28.825856 (+ 3718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:28.826885 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:28.826893 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:28.826896 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:28.827074 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:28.827531 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:28.827537 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:28.827548 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:28.827977 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:28.828255 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:28.828782 (+ 527us) server_negotiation.cc:299] Negotiation successful
0807 10:53:28.829045 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
Aug 07 10:53:28 dist-test-slave-x5fp krb5kdc[23487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100008, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.252@KRBTEST.COM for kudu/127.3.73.252@KRBTEST.COM
Aug 07 10:53:28 dist-test-slave-x5fp krb5kdc[23487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.253@KRBTEST.COM
I20260807 10:53:28.845046 23837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.829584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58602 (local address 127.3.73.252:36205)
0807 10:53:28.829768 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:28.829776 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:28.829825 (+ 49us) server_negotiation.cc:407] Connection header received
0807 10:53:28.829998 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:28.830003 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:28.830081 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:28.830190 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:28.831290 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.832383 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.833768 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.834193 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.834291 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:28.838062 (+ 3771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:28.838069 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:28.838073 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:28.838113 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:28.841588 (+ 3475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:28.842760 (+ 1172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:28.842764 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:28.842767 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:28.842944 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:28.843432 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:28.843440 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:28.843466 (+ 26us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:28.843867 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:28.844049 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:28.844387 (+ 338us) server_negotiation.cc:299] Negotiation successful
0807 10:53:28.844649 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 10:53:28.845252 23839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.829414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:36205 (local address 127.0.0.1:58602)
0807 10:53:28.829742 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:28.829763 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 10:53:28.829879 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:53:28.830277 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:53:28.830285 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:53:28.830903 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:53:28.831113 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:28.831124 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.832555 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.832559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:28.833605 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:28.833619 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.834348 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.834353 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:28.834507 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:28.835248 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:53:28.835267 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:53:28.837820 (+ 2553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:53:28.841751 (+ 3931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:28.841758 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:28.841762 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:28.842583 (+ 821us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:28.843089 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:28.843096 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:28.843103 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:28.843294 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:28.844009 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:53:28.844013 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:53:28.844146 (+ 133us) client_negotiation.cc:769] Sending connection context
0807 10:53:28.844302 (+ 156us) client_negotiation.cc:240] Negotiation successful
0807 10:53:28.844851 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":152}
I20260807 10:53:28.849902 23724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.831746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55194 (local address 127.3.73.253:43503)
0807 10:53:28.831922 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:28.831927 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:28.831946 (+ 19us) server_negotiation.cc:407] Connection header received
0807 10:53:28.832100 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:28.832104 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:28.832173 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:28.832264 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:28.834475 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.835211 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.836537 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.837120 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.837279 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:28.843314 (+ 6035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:28.843319 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:28.843323 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:28.843399 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:28.846675 (+ 3276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:28.847465 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:28.847473 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:28.847479 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:28.847650 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:28.848049 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:28.848054 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:28.848058 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:28.848423 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:28.848761 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:28.849108 (+ 347us) server_negotiation.cc:299] Negotiation successful
0807 10:53:28.849540 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 10:53:28.851840 23834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2e3ddb225d7433d9d0c65a0783f1943: Bootstrap starting.
I20260807 10:53:28.855795 3367 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 10:53:28.856153 23834 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2e3ddb225d7433d9d0c65a0783f1943: Neither blocks nor log segments found. Creating new log.
I20260807 10:53:28.857414 23834 log.cc:824] T 00000000000000000000000000000000 P a2e3ddb225d7433d9d0c65a0783f1943: Log is configured to *not* fsync() on all Append() calls
I20260807 10:53:28.860993 23834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2e3ddb225d7433d9d0c65a0783f1943: No bootstrap required, opened a new log
I20260807 10:53:28.876231 23834 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2e3ddb225d7433d9d0c65a0783f1943 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0800f1aa8ca4e5289677eb5c6cbe151" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45425 } } peers { permanent_uuid: "73399a33ca8a491983e03ba207b45b44" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 43503 } } peers { permanent_uuid: "a2e3ddb225d7433d9d0c65a0783f1943" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 36205 } }
I20260807 10:53:28.877002 23834 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2e3ddb225d7433d9d0c65a0783f1943 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:53:28.877229 23834 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2e3ddb225d7433d9d0c65a0783f1943 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2e3ddb225d7433d9d0c65a0783f1943, State: Initialized, Role: FOLLOWER
I20260807 10:53:28.878242 23834 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2e3ddb225d7433d9d0c65a0783f1943 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0800f1aa8ca4e5289677eb5c6cbe151" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45425 } } peers { permanent_uuid: "73399a33ca8a491983e03ba207b45b44" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 43503 } } peers { permanent_uuid: "a2e3ddb225d7433d9d0c65a0783f1943" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 36205 } }
I20260807 10:53:28.882246 23841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.861437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60894 (local address 127.3.73.254:45425)
0807 10:53:28.861662 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:28.861671 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:28.862136 (+ 465us) server_negotiation.cc:407] Connection header received
0807 10:53:28.862463 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:28.862469 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:28.862555 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:28.862616 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:28.864533 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.865711 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.868201 (+ 2490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.868903 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.869029 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:28.873245 (+ 4216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:28.873253 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:28.873258 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:28.873308 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:28.877742 (+ 4434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:28.878808 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:28.878816 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:28.878821 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:28.879012 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:28.879562 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:28.879570 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:28.879573 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:28.879978 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:28.880258 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:28.881539 (+ 1281us) server_negotiation.cc:299] Negotiation successful
0807 10:53:28.881846 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 10:53:28.883649 23850 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2e3ddb225d7433d9d0c65a0783f1943 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0800f1aa8ca4e5289677eb5c6cbe151" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45425 } } peers { permanent_uuid: "73399a33ca8a491983e03ba207b45b44" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 43503 } } peers { permanent_uuid: "a2e3ddb225d7433d9d0c65a0783f1943" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 36205 } } }
I20260807 10:53:28.884148 23850 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2e3ddb225d7433d9d0c65a0783f1943 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 10:53:28.884809 23724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.862286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55204 (local address 127.3.73.253:43503)
0807 10:53:28.862464 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:28.862470 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:28.865066 (+ 2596us) server_negotiation.cc:407] Connection header received
0807 10:53:28.865353 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:28.865359 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:28.865446 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:28.865515 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:28.868145 (+ 2630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.869244 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.871597 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.872459 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.872569 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:28.876714 (+ 4145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:28.876752 (+ 38us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:28.876758 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:28.876813 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:28.880502 (+ 3689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:28.881698 (+ 1196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:28.881704 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:28.881707 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:28.881873 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:28.882475 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:28.882484 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:28.882487 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:28.883325 (+ 838us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:28.883614 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:28.884260 (+ 646us) server_negotiation.cc:299] Negotiation successful
0807 10:53:28.884493 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 10:53:28.885623 23834 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2e3ddb225d7433d9d0c65a0783f1943 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 10:53:28.897172 23869 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a2e3ddb225d7433d9d0c65a0783f1943: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 10:53:28.897336 23869 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 10:53:28.899003 23837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.865203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58608 (local address 127.3.73.252:36205)
0807 10:53:28.865381 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:28.865387 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:28.876944 (+ 11557us) server_negotiation.cc:407] Connection header received
0807 10:53:28.877476 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:28.877481 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:28.877565 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:28.877639 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:28.880027 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.881574 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.885493 (+ 3919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.886254 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.886403 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:28.889762 (+ 3359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:28.889773 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:28.889780 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:28.889828 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:28.894076 (+ 4248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:28.895182 (+ 1106us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:28.895188 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:28.895191 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:28.895380 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:28.895916 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:28.895921 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:28.895925 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:28.896539 (+ 614us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:28.898135 (+ 1596us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:28.898297 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 10:53:28.898558 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":62}
I20260807 10:53:28.894654 23734 master_runner.cc:186] Error getting master registration for 127.3.73.254:45425: OK, instance_id { permanent_uuid: "e0800f1aa8ca4e5289677eb5c6cbe151" instance_seqno: 1786100007572270 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 10:53:28.905784 23734 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 10:53:28.908710 23716 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.252" port: 36205 } attempt: 5
Aug 07 10:53:28 dist-test-slave-x5fp krb5kdc[23487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100008, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.253@KRBTEST.COM for kudu/127.3.73.252@KRBTEST.COM
I20260807 10:53:28.924829 23721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.909479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:36205 (local address 127.0.0.1:58614)
0807 10:53:28.909671 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:28.909693 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 10:53:28.909817 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:53:28.910217 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:53:28.910222 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:53:28.910693 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:53:28.910871 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:28.910880 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.912038 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.912044 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:28.913510 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:28.913525 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.914390 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.914395 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:28.914625 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:28.915336 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:53:28.915354 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:53:28.917692 (+ 2338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:53:28.921966 (+ 4274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:28.921972 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:28.921977 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:28.922414 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:28.922864 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:28.922869 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:28.922873 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:28.923022 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:28.923657 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:53:28.923662 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:53:28.923887 (+ 225us) client_negotiation.cc:769] Sending connection context
0807 10:53:28.924041 (+ 154us) client_negotiation.cc:240] Negotiation successful
0807 10:53:28.924378 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 10:53:28.924825 23837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:28.909649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58614 (local address 127.3.73.252:36205)
0807 10:53:28.909882 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:28.909891 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:28.909918 (+ 27us) server_negotiation.cc:407] Connection header received
0807 10:53:28.909985 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:28.909988 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:28.910068 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:28.910180 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:28.911033 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.911890 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.913697 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:28.914200 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:28.914380 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:28.917916 (+ 3536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:28.917929 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:28.917933 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:28.917970 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:28.921818 (+ 3848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:28.922541 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:28.922549 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:28.922553 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:28.922748 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:28.923148 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:28.923153 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:28.923157 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:28.923525 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:28.923696 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:28.924178 (+ 482us) server_negotiation.cc:299] Negotiation successful
0807 10:53:28.924402 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 10:53:28.930104 23716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73399a33ca8a491983e03ba207b45b44: Bootstrap starting.
I20260807 10:53:28.934063 23716 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73399a33ca8a491983e03ba207b45b44: Neither blocks nor log segments found. Creating new log.
I20260807 10:53:28.935197 23716 log.cc:824] T 00000000000000000000000000000000 P 73399a33ca8a491983e03ba207b45b44: Log is configured to *not* fsync() on all Append() calls
I20260807 10:53:28.938138 23716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73399a33ca8a491983e03ba207b45b44: No bootstrap required, opened a new log
I20260807 10:53:28.946460 23716 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73399a33ca8a491983e03ba207b45b44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0800f1aa8ca4e5289677eb5c6cbe151" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45425 } } peers { permanent_uuid: "73399a33ca8a491983e03ba207b45b44" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 43503 } } peers { permanent_uuid: "a2e3ddb225d7433d9d0c65a0783f1943" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 36205 } }
I20260807 10:53:28.946879 23716 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73399a33ca8a491983e03ba207b45b44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:53:28.947018 23716 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73399a33ca8a491983e03ba207b45b44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73399a33ca8a491983e03ba207b45b44, State: Initialized, Role: FOLLOWER
I20260807 10:53:28.947690 23716 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73399a33ca8a491983e03ba207b45b44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0800f1aa8ca4e5289677eb5c6cbe151" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45425 } } peers { permanent_uuid: "73399a33ca8a491983e03ba207b45b44" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 43503 } } peers { permanent_uuid: "a2e3ddb225d7433d9d0c65a0783f1943" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 36205 } }
I20260807 10:53:28.948789 23874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73399a33ca8a491983e03ba207b45b44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0800f1aa8ca4e5289677eb5c6cbe151" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45425 } } peers { permanent_uuid: "73399a33ca8a491983e03ba207b45b44" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 43503 } } peers { permanent_uuid: "a2e3ddb225d7433d9d0c65a0783f1943" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 36205 } } }
I20260807 10:53:28.949223 23874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73399a33ca8a491983e03ba207b45b44 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 10:53:28.950055 23716 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73399a33ca8a491983e03ba207b45b44 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:53:28.958704 23617 master_runner.cc:186] Error getting master registration for 127.3.73.254:45425: OK, instance_id { permanent_uuid: "e0800f1aa8ca4e5289677eb5c6cbe151" instance_seqno: 1786100007572270 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 10:53:28.971717 23617 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 10:53:28.971906 23891 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 73399a33ca8a491983e03ba207b45b44: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 10:53:28.972048 23891 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260807 10:53:29.000833 23768 tablet.cc:2369] T 00000000000000000000000000000000 P a2e3ddb225d7433d9d0c65a0783f1943: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:53:29.007990 23608 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.252" port: 36205 } attempt: 5
Aug 07 10:53:29 dist-test-slave-x5fp krb5kdc[23487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100007, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for kudu/127.3.73.252@KRBTEST.COM
I20260807 10:53:29.029160 23611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:29.008792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:36205 (local address 127.0.0.1:58620)
0807 10:53:29.009106 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:29.009129 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 10:53:29.009272 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:53:29.009701 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:53:29.009709 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:53:29.010270 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:53:29.010513 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:29.010527 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:29.012056 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:29.012062 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:29.013696 (+ 1634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:29.013716 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:29.014904 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:29.014911 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:29.015105 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:29.015926 (+ 821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:53:29.015953 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:53:29.019195 (+ 3242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:53:29.025625 (+ 6430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:29.025632 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:29.025637 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:29.026283 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:29.026818 (+ 535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:29.026823 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:29.026827 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:29.027053 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:29.027824 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:53:29.027830 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:53:29.028012 (+ 182us) client_negotiation.cc:769] Sending connection context
0807 10:53:29.028194 (+ 182us) client_negotiation.cc:240] Negotiation successful
0807 10:53:29.028695 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172}
I20260807 10:53:29.029527 23837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:29.008887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58620 (local address 127.3.73.252:36205)
0807 10:53:29.009095 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:29.009103 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:29.009223 (+ 120us) server_negotiation.cc:407] Connection header received
0807 10:53:29.009414 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:29.009420 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:29.009509 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:29.009632 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:29.010694 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:29.011867 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:29.013906 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:29.014711 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:29.014851 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:29.019973 (+ 5122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:29.019980 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:29.019986 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:29.020036 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:29.025405 (+ 5369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:29.026469 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:29.026476 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:29.026481 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:29.026671 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:29.027199 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:29.027209 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:29.027213 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:29.027650 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:29.027861 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:29.028368 (+ 507us) server_negotiation.cc:299] Negotiation successful
0807 10:53:29.028657 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 10:53:29.035943 23608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0800f1aa8ca4e5289677eb5c6cbe151: Bootstrap starting.
I20260807 10:53:29.041951 23608 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0800f1aa8ca4e5289677eb5c6cbe151: Neither blocks nor log segments found. Creating new log.
I20260807 10:53:29.043500 23608 log.cc:824] T 00000000000000000000000000000000 P e0800f1aa8ca4e5289677eb5c6cbe151: Log is configured to *not* fsync() on all Append() calls
I20260807 10:53:29.047829 23608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0800f1aa8ca4e5289677eb5c6cbe151: No bootstrap required, opened a new log
I20260807 10:53:29.063295 23608 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0800f1aa8ca4e5289677eb5c6cbe151 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0800f1aa8ca4e5289677eb5c6cbe151" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45425 } } peers { permanent_uuid: "73399a33ca8a491983e03ba207b45b44" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 43503 } } peers { permanent_uuid: "a2e3ddb225d7433d9d0c65a0783f1943" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 36205 } }
I20260807 10:53:29.063848 23608 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0800f1aa8ca4e5289677eb5c6cbe151 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:53:29.063997 23608 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0800f1aa8ca4e5289677eb5c6cbe151 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0800f1aa8ca4e5289677eb5c6cbe151, State: Initialized, Role: FOLLOWER
I20260807 10:53:29.064843 23608 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0800f1aa8ca4e5289677eb5c6cbe151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0800f1aa8ca4e5289677eb5c6cbe151" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45425 } } peers { permanent_uuid: "73399a33ca8a491983e03ba207b45b44" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 43503 } } peers { permanent_uuid: "a2e3ddb225d7433d9d0c65a0783f1943" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 36205 } }
I20260807 10:53:29.066051 23894 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0800f1aa8ca4e5289677eb5c6cbe151 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0800f1aa8ca4e5289677eb5c6cbe151" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45425 } } peers { permanent_uuid: "73399a33ca8a491983e03ba207b45b44" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 43503 } } peers { permanent_uuid: "a2e3ddb225d7433d9d0c65a0783f1943" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 36205 } } }
I20260807 10:53:29.066563 23894 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0800f1aa8ca4e5289677eb5c6cbe151 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 10:53:29.067714 23608 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0800f1aa8ca4e5289677eb5c6cbe151 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:53:29.088862 23503 master_runner.cc:263] No leader master found from master e0800f1aa8ca4e5289677eb5c6cbe151
I20260807 10:53:29.089076 23503 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 10:53:29.095893 23907 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e0800f1aa8ca4e5289677eb5c6cbe151: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 10:53:29.096081 23907 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260807 10:53:29.154606 23655 tablet.cc:2369] T 00000000000000000000000000000000 P 73399a33ca8a491983e03ba207b45b44: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:53:29.240841 23894 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0800f1aa8ca4e5289677eb5c6cbe151 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 10:53:29.241222 23894 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0800f1aa8ca4e5289677eb5c6cbe151 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0800f1aa8ca4e5289677eb5c6cbe151" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45425 } } peers { permanent_uuid: "73399a33ca8a491983e03ba207b45b44" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 43503 } } peers { permanent_uuid: "a2e3ddb225d7433d9d0c65a0783f1943" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 36205 } }
I20260807 10:53:29.244707 23684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e0800f1aa8ca4e5289677eb5c6cbe151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73399a33ca8a491983e03ba207b45b44" is_pre_election: true
I20260807 10:53:29.245438 23684 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 73399a33ca8a491983e03ba207b45b44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0800f1aa8ca4e5289677eb5c6cbe151 in term 0.
I20260807 10:53:29.247032 23801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e0800f1aa8ca4e5289677eb5c6cbe151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2e3ddb225d7433d9d0c65a0783f1943" is_pre_election: true
I20260807 10:53:29.247628 23801 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a2e3ddb225d7433d9d0c65a0783f1943 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0800f1aa8ca4e5289677eb5c6cbe151 in term 0.
I20260807 10:53:29.243081 23894 leader_election.cc:288] T 00000000000000000000000000000000 P e0800f1aa8ca4e5289677eb5c6cbe151 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73399a33ca8a491983e03ba207b45b44 (127.3.73.253:43503), a2e3ddb225d7433d9d0c65a0783f1943 (127.3.73.252:36205)
I20260807 10:53:29.246299 23530 leader_election.cc:302] T 00000000000000000000000000000000 P e0800f1aa8ca4e5289677eb5c6cbe151 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73399a33ca8a491983e03ba207b45b44, e0800f1aa8ca4e5289677eb5c6cbe151; no voters:
I20260807 10:53:29.249505 23894 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0800f1aa8ca4e5289677eb5c6cbe151 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 10:53:29.249689 23894 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0800f1aa8ca4e5289677eb5c6cbe151 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 10:53:29.249800 23894 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0800f1aa8ca4e5289677eb5c6cbe151 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:53:29.259135 23894 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0800f1aa8ca4e5289677eb5c6cbe151 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0800f1aa8ca4e5289677eb5c6cbe151" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45425 } } peers { permanent_uuid: "73399a33ca8a491983e03ba207b45b44" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 43503 } } peers { permanent_uuid: "a2e3ddb225d7433d9d0c65a0783f1943" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 36205 } }
I20260807 10:53:29.262902 23684 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e0800f1aa8ca4e5289677eb5c6cbe151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73399a33ca8a491983e03ba207b45b44"
I20260807 10:53:29.263185 23684 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73399a33ca8a491983e03ba207b45b44 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:53:29.267154 23684 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 73399a33ca8a491983e03ba207b45b44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0800f1aa8ca4e5289677eb5c6cbe151 in term 1.
I20260807 10:53:29.267910 23530 leader_election.cc:302] T 00000000000000000000000000000000 P e0800f1aa8ca4e5289677eb5c6cbe151 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73399a33ca8a491983e03ba207b45b44, e0800f1aa8ca4e5289677eb5c6cbe151; no voters:
I20260807 10:53:29.271433 23894 leader_election.cc:288] T 00000000000000000000000000000000 P e0800f1aa8ca4e5289677eb5c6cbe151 [CANDIDATE]: Term 1 election: Requested vote from peers 73399a33ca8a491983e03ba207b45b44 (127.3.73.253:43503), a2e3ddb225d7433d9d0c65a0783f1943 (127.3.73.252:36205)
I20260807 10:53:29.272183 23894 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0800f1aa8ca4e5289677eb5c6cbe151 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:53:29.272713 23894 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0800f1aa8ca4e5289677eb5c6cbe151 [term 1 LEADER]: Becoming Leader. State: Replica: e0800f1aa8ca4e5289677eb5c6cbe151, State: Running, Role: LEADER
I20260807 10:53:29.273682 23894 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0800f1aa8ca4e5289677eb5c6cbe151 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0800f1aa8ca4e5289677eb5c6cbe151" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45425 } } peers { permanent_uuid: "73399a33ca8a491983e03ba207b45b44" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 43503 } } peers { permanent_uuid: "a2e3ddb225d7433d9d0c65a0783f1943" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 36205 } }
I20260807 10:53:29.281948 23801 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e0800f1aa8ca4e5289677eb5c6cbe151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2e3ddb225d7433d9d0c65a0783f1943"
I20260807 10:53:29.282328 23801 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2e3ddb225d7433d9d0c65a0783f1943 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:53:29.291381 23801 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a2e3ddb225d7433d9d0c65a0783f1943 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0800f1aa8ca4e5289677eb5c6cbe151 in term 1.
I20260807 10:53:29.299516 23909 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0800f1aa8ca4e5289677eb5c6cbe151 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0800f1aa8ca4e5289677eb5c6cbe151. Latest consensus state: current_term: 1 leader_uuid: "e0800f1aa8ca4e5289677eb5c6cbe151" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0800f1aa8ca4e5289677eb5c6cbe151" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45425 } } peers { permanent_uuid: "73399a33ca8a491983e03ba207b45b44" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 43503 } } peers { permanent_uuid: "a2e3ddb225d7433d9d0c65a0783f1943" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 36205 } } }
I20260807 10:53:29.300220 23909 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0800f1aa8ca4e5289677eb5c6cbe151 [sys.catalog]: This master's current role is: LEADER
I20260807 10:53:29.307521 23914 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:53:29.310356 23914 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260807 10:53:29.310737 23539 tablet.cc:2369] T 00000000000000000000000000000000 P e0800f1aa8ca4e5289677eb5c6cbe151: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:53:29.334129 23801 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a2e3ddb225d7433d9d0c65a0783f1943 [term 1 FOLLOWER]: Refusing update from remote peer e0800f1aa8ca4e5289677eb5c6cbe151: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:53:29.335744 23909 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e0800f1aa8ca4e5289677eb5c6cbe151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2e3ddb225d7433d9d0c65a0783f1943" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 36205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:53:29.342597 23684 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 73399a33ca8a491983e03ba207b45b44 [term 1 FOLLOWER]: Refusing update from remote peer e0800f1aa8ca4e5289677eb5c6cbe151: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:53:29.344008 23909 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e0800f1aa8ca4e5289677eb5c6cbe151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73399a33ca8a491983e03ba207b45b44" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 43503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:53:29.366030 23850 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2e3ddb225d7433d9d0c65a0783f1943 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0800f1aa8ca4e5289677eb5c6cbe151. Latest consensus state: current_term: 1 leader_uuid: "e0800f1aa8ca4e5289677eb5c6cbe151" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0800f1aa8ca4e5289677eb5c6cbe151" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45425 } } peers { permanent_uuid: "73399a33ca8a491983e03ba207b45b44" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 43503 } } peers { permanent_uuid: "a2e3ddb225d7433d9d0c65a0783f1943" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 36205 } } }
I20260807 10:53:29.366560 23850 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2e3ddb225d7433d9d0c65a0783f1943 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 10:53:29.378952 23915 mvcc.cc:205] Tried to move back new op lower bound from 7315865638215794688 to 7315865637999915008. Current Snapshot: MvccSnapshot[applied={T|T < 7315865638215794688}]
I20260807 10:53:29.386696 23850 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2e3ddb225d7433d9d0c65a0783f1943 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e0800f1aa8ca4e5289677eb5c6cbe151" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0800f1aa8ca4e5289677eb5c6cbe151" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45425 } } peers { permanent_uuid: "73399a33ca8a491983e03ba207b45b44" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 43503 } } peers { permanent_uuid: "a2e3ddb225d7433d9d0c65a0783f1943" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 36205 } } }
I20260807 10:53:29.387019 23850 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2e3ddb225d7433d9d0c65a0783f1943 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 10:53:29.398723 23894 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0800f1aa8ca4e5289677eb5c6cbe151 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e0800f1aa8ca4e5289677eb5c6cbe151" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0800f1aa8ca4e5289677eb5c6cbe151" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45425 } } peers { permanent_uuid: "73399a33ca8a491983e03ba207b45b44" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 43503 } } peers { permanent_uuid: "a2e3ddb225d7433d9d0c65a0783f1943" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 36205 } } }
I20260807 10:53:29.399185 23894 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0800f1aa8ca4e5289677eb5c6cbe151 [sys.catalog]: This master's current role is: LEADER
I20260807 10:53:29.400450 23914 catalog_manager.cc:1347] Generated new cluster ID: 47332a3c24384aa980cb6345e049c1e1
I20260807 10:53:29.400573 23914 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:53:29.435497 23874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73399a33ca8a491983e03ba207b45b44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0800f1aa8ca4e5289677eb5c6cbe151. Latest consensus state: current_term: 1 leader_uuid: "e0800f1aa8ca4e5289677eb5c6cbe151" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0800f1aa8ca4e5289677eb5c6cbe151" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45425 } } peers { permanent_uuid: "73399a33ca8a491983e03ba207b45b44" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 43503 } } peers { permanent_uuid: "a2e3ddb225d7433d9d0c65a0783f1943" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 36205 } } }
I20260807 10:53:29.435878 23874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73399a33ca8a491983e03ba207b45b44 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 10:53:29.455304 23914 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:53:29.459270 23914 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:53:29.470299 23894 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0800f1aa8ca4e5289677eb5c6cbe151 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e0800f1aa8ca4e5289677eb5c6cbe151" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0800f1aa8ca4e5289677eb5c6cbe151" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45425 } } peers { permanent_uuid: "73399a33ca8a491983e03ba207b45b44" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 43503 } } peers { permanent_uuid: "a2e3ddb225d7433d9d0c65a0783f1943" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 36205 } } }
I20260807 10:53:29.470669 23894 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0800f1aa8ca4e5289677eb5c6cbe151 [sys.catalog]: This master's current role is: LEADER
I20260807 10:53:29.489585 23914 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0800f1aa8ca4e5289677eb5c6cbe151: Generated new TSK 0
I20260807 10:53:29.490767 23914 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 10:53:29.495514 23874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73399a33ca8a491983e03ba207b45b44 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e0800f1aa8ca4e5289677eb5c6cbe151" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0800f1aa8ca4e5289677eb5c6cbe151" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45425 } } peers { permanent_uuid: "73399a33ca8a491983e03ba207b45b44" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 43503 } } peers { permanent_uuid: "a2e3ddb225d7433d9d0c65a0783f1943" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 36205 } } }
I20260807 10:53:29.495872 23874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73399a33ca8a491983e03ba207b45b44 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 10:53:29.899822 23869 catalog_manager.cc:1259] Loaded cluster ID: 47332a3c24384aa980cb6345e049c1e1
I20260807 10:53:29.899993 23869 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a2e3ddb225d7433d9d0c65a0783f1943: loading cluster ID for follower catalog manager: success
I20260807 10:53:29.903090 23869 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a2e3ddb225d7433d9d0c65a0783f1943: acquiring CA information for follower catalog manager: success
I20260807 10:53:29.905254 23869 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a2e3ddb225d7433d9d0c65a0783f1943: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 10:53:29.974534 23891 catalog_manager.cc:1259] Loaded cluster ID: 47332a3c24384aa980cb6345e049c1e1
I20260807 10:53:29.974653 23891 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 73399a33ca8a491983e03ba207b45b44: loading cluster ID for follower catalog manager: success
I20260807 10:53:29.977653 23891 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 73399a33ca8a491983e03ba207b45b44: acquiring CA information for follower catalog manager: success
I20260807 10:53:29.979803 23891 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 73399a33ca8a491983e03ba207b45b44: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 10:53:30.077278 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 23503
I20260807 10:53:30.079141 23603 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:53:30.198938 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 23503
W20260807 10:53:30.233508 23758 connection.cc:535] server connection from 127.0.0.1:58620 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 10:53:30.233971 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 23617
I20260807 10:53:30.235364 23710 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:53:30.354444 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 23617
I20260807 10:53:30.382095 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 23734
I20260807 10:53:30.383208 23829 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:53:30.498314 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 23734
2026-08-07T10:53:30Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5180 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:53:30 dist-test-slave-x5fp krb5kdc[23942](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:53:30 dist-test-slave-x5fp krb5kdc[23942](info): set up 2 sockets
Aug 07 10:53:30 dist-test-slave-x5fp krb5kdc[23942](info): commencing operation
krb5kdc: starting...
W20260807 10:53:31.874555 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.319s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:53:32 dist-test-slave-x5fp krb5kdc[23942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:53:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:53:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.73.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
I20260807 10:53:32.230722 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:36649
--webserver_interface=127.3.73.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:45491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:36649
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:53:32.609824 23963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:53:32.610248 23963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:53:32.610430 23963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:53:32.618212 23963 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:53:32.618399 23963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:53:32.618532 23963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:53:32.618643 23963 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:53:32.618726 23963 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:53:32.629535 23963 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:45491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:36649
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:36649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:53:32.631084 23963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:53:32.632532 23963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:53:32.646622 23974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:32.646781 23977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:32.646823 23973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:53:32.647977 23963 server_base.cc:1034] running on GCE node
I20260807 10:53:32.649120 23963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:53:32.650789 23963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:53:32.652072 23963 hybrid_clock.cc:648] HybridClock initialized: now 1786100012652035 us; error 64 us; skew 500 ppm
I20260807 10:53:32.652498 23963 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:53:32 dist-test-slave-x5fp krb5kdc[23942](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100012, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:53:32.656337 23963 init.cc:377] Logged in from keytab as kudu/127.3.73.254@KRBTEST.COM (short username kudu)
I20260807 10:53:32.658215 23963 webserver.cc:460] Webserver started at http://127.3.73.254:37965/ using document root <none> and password file <none>
I20260807 10:53:32.658996 23963 fs_manager.cc:359] Metadata directory not provided
I20260807 10:53:32.659157 23963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:53:32.659500 23963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:53:32.662983 23963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "8601841593744cb79be3efdf40910fdc"
format_stamp: "Formatted at 2026-08-07 10:53:32 on dist-test-slave-x5fp"
server_key: "21d579856977c4ccf2727b0253e64a04"
server_key_iv: "fdc9bc9d61288fd5862a9e439d34146a"
server_key_version: "encryptionkey@0"
I20260807 10:53:32.663841 23963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "8601841593744cb79be3efdf40910fdc"
format_stamp: "Formatted at 2026-08-07 10:53:32 on dist-test-slave-x5fp"
server_key: "21d579856977c4ccf2727b0253e64a04"
server_key_iv: "fdc9bc9d61288fd5862a9e439d34146a"
server_key_version: "encryptionkey@0"
I20260807 10:53:32.669628 23963 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260807 10:53:32.673520 23989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:32.674978 23963 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 10:53:32.675223 23963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "8601841593744cb79be3efdf40910fdc"
format_stamp: "Formatted at 2026-08-07 10:53:32 on dist-test-slave-x5fp"
server_key: "21d579856977c4ccf2727b0253e64a04"
server_key_iv: "fdc9bc9d61288fd5862a9e439d34146a"
server_key_version: "encryptionkey@0"
I20260807 10:53:32.675482 23963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:53:32.695785 23963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:53:32.700505 23963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:53:32.700896 23963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:53:32.722097 23963 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:36649
I20260807 10:53:32.722153 24059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:36649 every 8 connection(s)
I20260807 10:53:32.723672 23963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:53:32.726699 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 23963
I20260807 10:53:32.726938 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:53:32.727669 3367 external_mini_cluster.cc:1428] Setting key 0bff53af435deee6d858512879cc602e
I20260807 10:53:32.727958 24060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:53:32.743739 24060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8601841593744cb79be3efdf40910fdc: Bootstrap starting.
Aug 07 10:53:32 dist-test-slave-x5fp krb5kdc[23942](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:53:32.748077 24060 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8601841593744cb79be3efdf40910fdc: Neither blocks nor log segments found. Creating new log.
I20260807 10:53:32.749619 24060 log.cc:824] T 00000000000000000000000000000000 P 8601841593744cb79be3efdf40910fdc: Log is configured to *not* fsync() on all Append() calls
I20260807 10:53:32.753670 24060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8601841593744cb79be3efdf40910fdc: No bootstrap required, opened a new log
I20260807 10:53:32.756294 24063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:32.729890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58600 (local address 127.3.73.254:36649)
0807 10:53:32.730771 (+ 881us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:32.730792 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:32.730858 (+ 66us) server_negotiation.cc:407] Connection header received
0807 10:53:32.733360 (+ 2502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:32.733398 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:32.734036 (+ 638us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:32.734550 (+ 514us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:32.736113 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:32.738154 (+ 2041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:32.739687 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:32.740900 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:32.741116 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:32.746042 (+ 4926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:32.746054 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:32.746082 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:32.746139 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:32.750582 (+ 4443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:32.751777 (+ 1195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:32.751787 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:32.751796 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:32.752029 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:32.752475 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:32.752481 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:32.752484 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:32.753332 (+ 848us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:32.753666 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:32.754344 (+ 678us) server_negotiation.cc:299] Negotiation successful
0807 10:53:32.754849 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":372,"threads_started":1}
I20260807 10:53:32.761204 3367 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 10:53:32.762948 24060 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8601841593744cb79be3efdf40910fdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8601841593744cb79be3efdf40910fdc" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 36649 } }
I20260807 10:53:32.763377 24060 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8601841593744cb79be3efdf40910fdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:53:32.763516 24060 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8601841593744cb79be3efdf40910fdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8601841593744cb79be3efdf40910fdc, State: Initialized, Role: FOLLOWER
I20260807 10:53:32.764117 24060 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8601841593744cb79be3efdf40910fdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8601841593744cb79be3efdf40910fdc" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 36649 } }
I20260807 10:53:32.764401 24060 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8601841593744cb79be3efdf40910fdc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:53:32.764580 24060 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8601841593744cb79be3efdf40910fdc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:53:32.764846 24060 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8601841593744cb79be3efdf40910fdc [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:53:32.767184 24060 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8601841593744cb79be3efdf40910fdc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8601841593744cb79be3efdf40910fdc" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 36649 } }
I20260807 10:53:32.767727 24060 leader_election.cc:302] T 00000000000000000000000000000000 P 8601841593744cb79be3efdf40910fdc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8601841593744cb79be3efdf40910fdc; no voters:
I20260807 10:53:32.768353 24060 leader_election.cc:288] T 00000000000000000000000000000000 P 8601841593744cb79be3efdf40910fdc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:53:32.768539 24065 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8601841593744cb79be3efdf40910fdc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:53:32.769295 24065 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8601841593744cb79be3efdf40910fdc [term 1 LEADER]: Becoming Leader. State: Replica: 8601841593744cb79be3efdf40910fdc, State: Running, Role: LEADER
I20260807 10:53:32.769886 24065 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8601841593744cb79be3efdf40910fdc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8601841593744cb79be3efdf40910fdc" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 36649 } }
I20260807 10:53:32.770718 24060 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8601841593744cb79be3efdf40910fdc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:53:32.772680 24066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8601841593744cb79be3efdf40910fdc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8601841593744cb79be3efdf40910fdc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8601841593744cb79be3efdf40910fdc" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 36649 } } }
I20260807 10:53:32.772934 24067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8601841593744cb79be3efdf40910fdc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8601841593744cb79be3efdf40910fdc. Latest consensus state: current_term: 1 leader_uuid: "8601841593744cb79be3efdf40910fdc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8601841593744cb79be3efdf40910fdc" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 36649 } } }
I20260807 10:53:32.773168 24066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8601841593744cb79be3efdf40910fdc [sys.catalog]: This master's current role is: LEADER
I20260807 10:53:32.773205 24067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8601841593744cb79be3efdf40910fdc [sys.catalog]: This master's current role is: LEADER
I20260807 10:53:32.776042 24074 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:53:32.782020 24074 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:53:32.791509 24074 catalog_manager.cc:1347] Generated new cluster ID: 77ea6269db0546398b52b0afbdc4a002
I20260807 10:53:32.791647 24074 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:53:32.823793 24074 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:53:32.825428 24074 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:53:32.843058 24074 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8601841593744cb79be3efdf40910fdc: Generated new TSK 0
I20260807 10:53:32.844069 24074 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 10:53:32.874891 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 23963
I20260807 10:53:32.876371 24054 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:53:32.978168 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 23963
2026-08-07T10:53:33Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2473 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:53:33 dist-test-slave-x5fp krb5kdc[24097](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:53:33 dist-test-slave-x5fp krb5kdc[24097](info): set up 2 sockets
Aug 07 10:53:33 dist-test-slave-x5fp krb5kdc[24097](info): commencing operation
krb5kdc: starting...
W20260807 10:53:34.378010 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.349s user 0.004s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:53:34 dist-test-slave-x5fp krb5kdc[24097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100014, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:53:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:53:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.73.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
I20260807 10:53:34.737582 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:35855
--webserver_interface=127.3.73.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:46781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:35855
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:53:35.128940 24115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:53:35.129349 24115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:53:35.129539 24115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:53:35.137362 24115 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:53:35.137495 24115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:53:35.137543 24115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:53:35.137594 24115 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:53:35.137640 24115 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:53:35.148130 24115 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:46781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:35855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:35855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:53:35.149658 24115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:53:35.151021 24115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:53:35.164150 24121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:35.164150 24127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:35.164172 24124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:53:35.170554 24115 server_base.cc:1034] running on GCE node
I20260807 10:53:35.171705 24115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:53:35.173856 24115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:53:35.175146 24115 hybrid_clock.cc:648] HybridClock initialized: now 1786100015175090 us; error 54 us; skew 500 ppm
I20260807 10:53:35.175602 24115 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:53:35 dist-test-slave-x5fp krb5kdc[24097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100015, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:53:35.179575 24115 init.cc:377] Logged in from keytab as kudu/127.3.73.254@KRBTEST.COM (short username kudu)
I20260807 10:53:35.181563 24115 webserver.cc:460] Webserver started at http://127.3.73.254:39523/ using document root <none> and password file <none>
I20260807 10:53:35.182371 24115 fs_manager.cc:359] Metadata directory not provided
I20260807 10:53:35.182518 24115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:53:35.182883 24115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:53:35.186684 24115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "e9c8d1c8814f4b03b923ff6cdc695c05"
format_stamp: "Formatted at 2026-08-07 10:53:35 on dist-test-slave-x5fp"
server_key: "4a762993e8207c6ceee90645f7869f5c"
server_key_iv: "8601664132f7a6639e84a2aaa65819e7"
server_key_version: "encryptionkey@0"
I20260807 10:53:35.187662 24115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "e9c8d1c8814f4b03b923ff6cdc695c05"
format_stamp: "Formatted at 2026-08-07 10:53:35 on dist-test-slave-x5fp"
server_key: "4a762993e8207c6ceee90645f7869f5c"
server_key_iv: "8601664132f7a6639e84a2aaa65819e7"
server_key_version: "encryptionkey@0"
I20260807 10:53:35.194115 24115 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 10:53:35.198784 24138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:35.200353 24115 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 10:53:35.200573 24115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "e9c8d1c8814f4b03b923ff6cdc695c05"
format_stamp: "Formatted at 2026-08-07 10:53:35 on dist-test-slave-x5fp"
server_key: "4a762993e8207c6ceee90645f7869f5c"
server_key_iv: "8601664132f7a6639e84a2aaa65819e7"
server_key_version: "encryptionkey@0"
I20260807 10:53:35.200901 24115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:53:35.218240 24115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:53:35.224335 24115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:53:35.224828 24115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:53:35.249624 24115 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:35855
I20260807 10:53:35.249665 24214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:35855 every 8 connection(s)
I20260807 10:53:35.251209 24115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:53:35.254657 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 24115
I20260807 10:53:35.254953 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:53:35.255697 3367 external_mini_cluster.cc:1428] Setting key 605c03b9c20a5646c4c32c6fddacb576
I20260807 10:53:35.255872 24215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:53:35.274258 24215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05: Bootstrap starting.
Aug 07 10:53:35 dist-test-slave-x5fp krb5kdc[24097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100014, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:53:35.278043 24215 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05: Neither blocks nor log segments found. Creating new log.
I20260807 10:53:35.279299 24215 log.cc:824] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05: Log is configured to *not* fsync() on all Append() calls
I20260807 10:53:35.282872 24215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05: No bootstrap required, opened a new log
I20260807 10:53:35.284454 24219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:35.258144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36214 (local address 127.3.73.254:35855)
0807 10:53:35.259180 (+ 1036us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:35.259197 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:35.259244 (+ 47us) server_negotiation.cc:407] Connection header received
0807 10:53:35.261213 (+ 1969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:35.261240 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:35.261729 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:35.262122 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:35.264472 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:35.266412 (+ 1940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:35.268558 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:35.269752 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:35.269983 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:35.276052 (+ 6069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:35.276063 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:35.276079 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:35.276132 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:35.279702 (+ 3570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:35.280596 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:35.280604 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:35.280611 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:35.280883 (+ 272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:35.281341 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:35.281346 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:35.281350 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:35.281987 (+ 637us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:35.282364 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:35.282830 (+ 466us) server_negotiation.cc:299] Negotiation successful
0807 10:53:35.283368 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":773,"thread_start_us":464,"threads_started":1}
I20260807 10:53:35.293233 24215 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9c8d1c8814f4b03b923ff6cdc695c05" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 35855 } }
I20260807 10:53:35.293736 24215 raft_consensus.cc:374] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:53:35.293871 24215 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9c8d1c8814f4b03b923ff6cdc695c05, State: Initialized, Role: FOLLOWER
I20260807 10:53:35.294574 24215 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9c8d1c8814f4b03b923ff6cdc695c05" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 35855 } }
I20260807 10:53:35.294866 24215 raft_consensus.cc:388] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:53:35.295027 24215 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:53:35.295222 24215 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:53:35.297704 24215 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9c8d1c8814f4b03b923ff6cdc695c05" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 35855 } }
I20260807 10:53:35.298507 24215 leader_election.cc:302] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9c8d1c8814f4b03b923ff6cdc695c05; no voters:
I20260807 10:53:35.299425 24215 leader_election.cc:288] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:53:35.299556 24222 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:53:35.300462 24222 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [term 1 LEADER]: Becoming Leader. State: Replica: e9c8d1c8814f4b03b923ff6cdc695c05, State: Running, Role: LEADER
I20260807 10:53:35.301529 24222 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9c8d1c8814f4b03b923ff6cdc695c05" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 35855 } }
I20260807 10:53:35.302361 24215 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:53:35.304942 24224 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e9c8d1c8814f4b03b923ff6cdc695c05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9c8d1c8814f4b03b923ff6cdc695c05" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 35855 } } }
I20260807 10:53:35.305595 24224 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [sys.catalog]: This master's current role is: LEADER
I20260807 10:53:35.305246 24225 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9c8d1c8814f4b03b923ff6cdc695c05. Latest consensus state: current_term: 1 leader_uuid: "e9c8d1c8814f4b03b923ff6cdc695c05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9c8d1c8814f4b03b923ff6cdc695c05" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 35855 } } }
I20260807 10:53:35.308347 24225 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [sys.catalog]: This master's current role is: LEADER
I20260807 10:53:35.311748 24229 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:53:35.318715 24229 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:53:35.329469 24229 catalog_manager.cc:1347] Generated new cluster ID: e4f39d529d4e4402bd5703345d6a4a30
I20260807 10:53:35.329664 24229 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:53:35.338886 24229 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:53:35.340459 24229 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:53:35.358369 24229 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05: Generated new TSK 0
I20260807 10:53:35.359565 24229 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.73.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab.
Entry for principal kudu/127.3.73.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab.
I20260807 10:53:35.450803 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.193:0
--local_ip_for_outbound_sockets=127.3.73.193
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:35855
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:46781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:53:35.830456 24246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:53:35.830787 24246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:53:35.830955 24246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:53:35.839956 24246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:53:35.840111 24246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:53:35.840327 24246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.193
I20260807 10:53:35.851397 24246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:46781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:35855
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.193
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:53:35.853106 24246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:53:35.854673 24246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:53:35.869215 24252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:35.869335 24255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:35.869354 24251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:53:35.870687 24246 server_base.cc:1034] running on GCE node
I20260807 10:53:35.871495 24246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:53:35.872632 24246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:53:35.873941 24246 hybrid_clock.cc:648] HybridClock initialized: now 1786100015873835 us; error 60 us; skew 500 ppm
I20260807 10:53:35.874353 24246 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:53:35 dist-test-slave-x5fp krb5kdc[24097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100015, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:53:35.884547 24246 init.cc:377] Logged in from keytab as kudu/127.3.73.193@KRBTEST.COM (short username kudu)
I20260807 10:53:35.886545 24246 webserver.cc:460] Webserver started at http://127.3.73.193:45273/ using document root <none> and password file <none>
I20260807 10:53:35.887396 24246 fs_manager.cc:359] Metadata directory not provided
I20260807 10:53:35.887543 24246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:53:35.887914 24246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:53:35.891559 24246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/data/instance:
uuid: "3425beb307af44598102bae102debd71"
format_stamp: "Formatted at 2026-08-07 10:53:35 on dist-test-slave-x5fp"
server_key: "7d2261ea05a9dba3968af1cd990cb68d"
server_key_iv: "57de6d69b5f35dacf8f12537c6805b32"
server_key_version: "encryptionkey@0"
I20260807 10:53:35.892549 24246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance:
uuid: "3425beb307af44598102bae102debd71"
format_stamp: "Formatted at 2026-08-07 10:53:35 on dist-test-slave-x5fp"
server_key: "7d2261ea05a9dba3968af1cd990cb68d"
server_key_iv: "57de6d69b5f35dacf8f12537c6805b32"
server_key_version: "encryptionkey@0"
I20260807 10:53:35.898794 24246 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 10:53:35.903329 24266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:35.905167 24246 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 10:53:35.905397 24246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/wal
uuid: "3425beb307af44598102bae102debd71"
format_stamp: "Formatted at 2026-08-07 10:53:35 on dist-test-slave-x5fp"
server_key: "7d2261ea05a9dba3968af1cd990cb68d"
server_key_iv: "57de6d69b5f35dacf8f12537c6805b32"
server_key_version: "encryptionkey@0"
I20260807 10:53:35.905704 24246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:53:35.923430 24246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:53:35.928817 24246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:53:35.929171 24246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:53:35.930318 24246 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:53:35.931989 24246 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:53:35.932101 24246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:35.932283 24246 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:53:35.932375 24246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:35.980847 24246 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.193:44389
I20260807 10:53:35.980947 24423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.193:44389 every 8 connection(s)
I20260807 10:53:35.982262 24246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
I20260807 10:53:35.988135 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 24246
I20260807 10:53:35.988328 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance
I20260807 10:53:35.989248 3367 external_mini_cluster.cc:1428] Setting key 57084bc02f83f189bca0dbe7b3269ca7
Aug 07 10:53:35 dist-test-slave-x5fp krb5kdc[24097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100015, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:53:36.007035 24428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:35.985006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:43373 (local address 127.3.73.254:35855)
0807 10:53:35.985492 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:35.985497 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:35.986529 (+ 1032us) server_negotiation.cc:407] Connection header received
0807 10:53:35.988606 (+ 2077us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:35.988614 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:35.988740 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:35.988880 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:35.990667 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:35.991808 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:35.993549 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:35.994398 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:35.994573 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:35.998352 (+ 3779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:35.998363 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:35.998372 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:35.998426 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:36.002578 (+ 4152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:36.003730 (+ 1152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:36.003739 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:36.003746 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:36.003950 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:36.004475 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:36.004483 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:36.004490 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:36.004963 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:36.005142 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:36.006284 (+ 1142us) server_negotiation.cc:299] Negotiation successful
0807 10:53:36.006643 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":243,"threads_started":1}
I20260807 10:53:36.008311 24429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:35.985578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.193:43373)
0807 10:53:35.986353 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:35.986396 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 10:53:35.988135 (+ 1739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:53:35.989132 (+ 997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:53:35.989141 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:53:35.989746 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:53:35.990487 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:35.990506 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:35.991989 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:35.991992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:35.993375 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:35.993390 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:35.994624 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:35.994629 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:35.994832 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:35.995643 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:53:35.995667 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:53:35.998060 (+ 2393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:53:36.002861 (+ 4801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:36.002871 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:36.002886 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:36.003560 (+ 674us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:36.004090 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:36.004094 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:36.004097 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:36.004333 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:36.005127 (+ 794us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:53:36.005134 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:53:36.005543 (+ 409us) client_negotiation.cc:769] Sending connection context
0807 10:53:36.006115 (+ 572us) client_negotiation.cc:240] Negotiation successful
0807 10:53:36.006639 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":314,"threads_started":1}
I20260807 10:53:36.010154 24424 heartbeater.cc:347] Connected to a master server at 127.3.73.254:35855
I20260807 10:53:36.010531 24424 heartbeater.cc:464] Registering TS with master...
I20260807 10:53:36.011510 24424 heartbeater.cc:511] Master 127.3.73.254:35855 requested a full tablet report, sending...
I20260807 10:53:36.014070 24161 ts_manager.cc:194] Registered new tserver with Master: 3425beb307af44598102bae102debd71 (127.3.73.193:44389)
I20260807 10:53:36.015636 24161 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.193@KRBTEST.COM'} at 127.3.73.193:43373
WARNING: no policy specified for kudu/127.3.73.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab.
Entry for principal kudu/127.3.73.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab.
I20260807 10:53:36.065536 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.194:0
--local_ip_for_outbound_sockets=127.3.73.194
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:35855
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:46781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:53:36.444491 24433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:53:36.444890 24433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:53:36.445087 24433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:53:36.452661 24433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:53:36.452881 24433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:53:36.453089 24433 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.194
I20260807 10:53:36.463784 24433 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:46781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:35855
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.194
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:53:36.465325 24433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:53:36.466810 24433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:53:36.481630 24439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:36.481680 24442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:36.481758 24440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:53:36.483107 24433 server_base.cc:1034] running on GCE node
I20260807 10:53:36.483862 24433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:53:36.485088 24433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:53:36.486351 24433 hybrid_clock.cc:648] HybridClock initialized: now 1786100016486306 us; error 39 us; skew 500 ppm
I20260807 10:53:36.486737 24433 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:53:36 dist-test-slave-x5fp krb5kdc[24097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100016, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:53:36.494560 24433 init.cc:377] Logged in from keytab as kudu/127.3.73.194@KRBTEST.COM (short username kudu)
I20260807 10:53:36.496345 24433 webserver.cc:460] Webserver started at http://127.3.73.194:42997/ using document root <none> and password file <none>
I20260807 10:53:36.497263 24433 fs_manager.cc:359] Metadata directory not provided
I20260807 10:53:36.497442 24433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:53:36.497816 24433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:53:36.501497 24433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/data/instance:
uuid: "2ad1efccd76440a6b62e006eb4a5f83e"
format_stamp: "Formatted at 2026-08-07 10:53:36 on dist-test-slave-x5fp"
server_key: "a578eea0239e45fb35269d810a4448bc"
server_key_iv: "941c49b2167c89b10e5f364a357476b9"
server_key_version: "encryptionkey@0"
I20260807 10:53:36.502558 24433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance:
uuid: "2ad1efccd76440a6b62e006eb4a5f83e"
format_stamp: "Formatted at 2026-08-07 10:53:36 on dist-test-slave-x5fp"
server_key: "a578eea0239e45fb35269d810a4448bc"
server_key_iv: "941c49b2167c89b10e5f364a357476b9"
server_key_version: "encryptionkey@0"
I20260807 10:53:36.509271 24433 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260807 10:53:36.513939 24455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:36.515609 24433 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 10:53:36.515833 24433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/wal
uuid: "2ad1efccd76440a6b62e006eb4a5f83e"
format_stamp: "Formatted at 2026-08-07 10:53:36 on dist-test-slave-x5fp"
server_key: "a578eea0239e45fb35269d810a4448bc"
server_key_iv: "941c49b2167c89b10e5f364a357476b9"
server_key_version: "encryptionkey@0"
I20260807 10:53:36.516077 24433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:53:36.530789 24433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:53:36.535799 24433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:53:36.536142 24433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:53:36.537283 24433 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:53:36.538873 24433 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:53:36.538998 24433 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:36.539157 24433 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:53:36.539238 24433 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:36.577430 24433 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.194:32827
I20260807 10:53:36.577481 24613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.194:32827 every 8 connection(s)
I20260807 10:53:36.578934 24433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
I20260807 10:53:36.581737 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 24433
I20260807 10:53:36.581995 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance
I20260807 10:53:36.582707 3367 external_mini_cluster.cc:1428] Setting key 8f52c48a09b46fd11f0cb7ab206e6296
Aug 07 10:53:36 dist-test-slave-x5fp krb5kdc[24097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100016, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:53:36.612175 24618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:36.582696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.194:39823)
0807 10:53:36.583607 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:36.583658 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 10:53:36.586301 (+ 2643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:53:36.587636 (+ 1335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:53:36.587655 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:53:36.588788 (+ 1133us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:53:36.589953 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:36.589991 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:36.591856 (+ 1865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:36.591862 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:36.594335 (+ 2473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:36.594356 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:36.595633 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:36.595643 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:36.596036 (+ 393us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:36.596936 (+ 900us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:53:36.596963 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:53:36.600348 (+ 3385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:53:36.605529 (+ 5181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:36.605545 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:36.605570 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:36.606472 (+ 902us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:36.607107 (+ 635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:36.607115 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:36.607119 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:36.607460 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:36.608271 (+ 811us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:53:36.608284 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:53:36.609041 (+ 757us) client_negotiation.cc:769] Sending connection context
0807 10:53:36.609501 (+ 460us) client_negotiation.cc:240] Negotiation successful
0807 10:53:36.610536 (+ 1035us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":415,"threads_started":1}
I20260807 10:53:36.613662 24617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:36.582116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:39823 (local address 127.3.73.254:35855)
0807 10:53:36.582689 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:36.582699 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:36.583779 (+ 1080us) server_negotiation.cc:407] Connection header received
0807 10:53:36.586774 (+ 2995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:36.586782 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:36.586881 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:36.586991 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:36.590243 (+ 3252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:36.591613 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:36.594557 (+ 2944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:36.595341 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:36.595512 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:36.600760 (+ 5248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:36.600770 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:36.600778 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:36.600834 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:36.605252 (+ 4418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:36.606657 (+ 1405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:36.606733 (+ 76us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:36.606740 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:36.606937 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:36.607634 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:36.607642 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:36.607648 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:36.608095 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:36.608257 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:36.612927 (+ 4670us) server_negotiation.cc:299] Negotiation successful
0807 10:53:36.613231 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":262,"threads_started":1}
I20260807 10:53:36.615342 24614 heartbeater.cc:347] Connected to a master server at 127.3.73.254:35855
I20260807 10:53:36.615722 24614 heartbeater.cc:464] Registering TS with master...
I20260807 10:53:36.616564 24614 heartbeater.cc:511] Master 127.3.73.254:35855 requested a full tablet report, sending...
I20260807 10:53:36.618737 24161 ts_manager.cc:194] Registered new tserver with Master: 2ad1efccd76440a6b62e006eb4a5f83e (127.3.73.194:32827)
I20260807 10:53:36.620074 24161 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.194@KRBTEST.COM'} at 127.3.73.194:39823
WARNING: no policy specified for kudu/127.3.73.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab.
Entry for principal kudu/127.3.73.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab.
I20260807 10:53:36.673496 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.195:0
--local_ip_for_outbound_sockets=127.3.73.195
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:35855
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:46781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 10:53:37.019042 24424 heartbeater.cc:503] Master 127.3.73.254:35855 was elected leader, sending a full tablet report...
W20260807 10:53:37.045984 24623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:53:37.046360 24623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:53:37.046538 24623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:53:37.054154 24623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:53:37.054348 24623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:53:37.054543 24623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.195
I20260807 10:53:37.065054 24623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:46781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:35855
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.195
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:53:37.066563 24623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:53:37.068140 24623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:53:37.083218 24629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:37.083259 24628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:37.083328 24632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:53:37.084499 24623 server_base.cc:1034] running on GCE node
I20260807 10:53:37.085399 24623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:53:37.086589 24623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:53:37.087884 24623 hybrid_clock.cc:648] HybridClock initialized: now 1786100017087829 us; error 79 us; skew 500 ppm
I20260807 10:53:37.088289 24623 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:53:37 dist-test-slave-x5fp krb5kdc[24097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100017, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:53:37.092021 24623 init.cc:377] Logged in from keytab as kudu/127.3.73.195@KRBTEST.COM (short username kudu)
I20260807 10:53:37.093780 24623 webserver.cc:460] Webserver started at http://127.3.73.195:44503/ using document root <none> and password file <none>
I20260807 10:53:37.094535 24623 fs_manager.cc:359] Metadata directory not provided
I20260807 10:53:37.094640 24623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:53:37.094913 24623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:53:37.098155 24623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/data/instance:
uuid: "be67427905c64a048b6def3957fdb418"
format_stamp: "Formatted at 2026-08-07 10:53:37 on dist-test-slave-x5fp"
server_key: "27791662d39c92f7c3f5fab83b1958db"
server_key_iv: "b1b7b729ff3e0a847dc9dc6232aa9ee2"
server_key_version: "encryptionkey@0"
I20260807 10:53:37.098954 24623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance:
uuid: "be67427905c64a048b6def3957fdb418"
format_stamp: "Formatted at 2026-08-07 10:53:37 on dist-test-slave-x5fp"
server_key: "27791662d39c92f7c3f5fab83b1958db"
server_key_iv: "b1b7b729ff3e0a847dc9dc6232aa9ee2"
server_key_version: "encryptionkey@0"
I20260807 10:53:37.104902 24623 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 10:53:37.109340 24641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:37.111155 24623 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260807 10:53:37.111411 24623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/wal
uuid: "be67427905c64a048b6def3957fdb418"
format_stamp: "Formatted at 2026-08-07 10:53:37 on dist-test-slave-x5fp"
server_key: "27791662d39c92f7c3f5fab83b1958db"
server_key_iv: "b1b7b729ff3e0a847dc9dc6232aa9ee2"
server_key_version: "encryptionkey@0"
I20260807 10:53:37.111688 24623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:53:37.135811 24623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:53:37.140926 24623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:53:37.141422 24623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:53:37.142642 24623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:53:37.144181 24623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:53:37.144302 24623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:37.144465 24623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:53:37.144544 24623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:37.183131 24623 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.195:42781
I20260807 10:53:37.183187 24807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.195:42781 every 8 connection(s)
I20260807 10:53:37.184608 24623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
I20260807 10:53:37.188393 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 24623
I20260807 10:53:37.188621 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance
I20260807 10:53:37.189335 3367 external_mini_cluster.cc:1428] Setting key 0d533c48f9b6b8dde9dfd092113372f1
Aug 07 10:53:37 dist-test-slave-x5fp krb5kdc[24097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100017, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:53:37.210348 24812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:37.187760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:38421 (local address 127.3.73.254:35855)
0807 10:53:37.188381 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:37.188389 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:37.189639 (+ 1250us) server_negotiation.cc:407] Connection header received
0807 10:53:37.192479 (+ 2840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:37.192485 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:37.192592 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:37.192760 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:37.194890 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:37.195779 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:37.197820 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:37.198292 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:37.198456 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:37.202553 (+ 4097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:37.202558 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:37.202561 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:37.202600 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:37.206577 (+ 3977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:37.207671 (+ 1094us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:37.207676 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:37.207679 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:37.207830 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:37.208328 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:37.208333 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:37.208335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:37.208637 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:37.208846 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:37.209706 (+ 860us) server_negotiation.cc:299] Negotiation successful
0807 10:53:37.209983 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":305,"threads_started":1}
I20260807 10:53:37.211812 24814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:37.188242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.195:38421)
0807 10:53:37.189266 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:37.189341 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0807 10:53:37.192095 (+ 2754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:53:37.193106 (+ 1011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:53:37.193121 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:53:37.193980 (+ 859us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:53:37.194670 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:37.194699 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:37.196069 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:37.196076 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:37.197642 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:37.197657 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:37.198571 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:37.198579 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:37.198852 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:37.199584 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:53:37.199610 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:53:37.202210 (+ 2600us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:53:37.206797 (+ 4587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:37.206806 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:37.206824 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:37.207502 (+ 678us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:37.207956 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:37.207960 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:37.207963 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:37.208189 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:37.208836 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:53:37.208844 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:53:37.209240 (+ 396us) client_negotiation.cc:769] Sending connection context
0807 10:53:37.209520 (+ 280us) client_negotiation.cc:240] Negotiation successful
0807 10:53:37.210179 (+ 659us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"thread_start_us":399,"threads_started":1}
I20260807 10:53:37.214112 24808 heartbeater.cc:347] Connected to a master server at 127.3.73.254:35855
I20260807 10:53:37.214488 24808 heartbeater.cc:464] Registering TS with master...
I20260807 10:53:37.215552 24808 heartbeater.cc:511] Master 127.3.73.254:35855 requested a full tablet report, sending...
I20260807 10:53:37.217525 24161 ts_manager.cc:194] Registered new tserver with Master: be67427905c64a048b6def3957fdb418 (127.3.73.195:42781)
I20260807 10:53:37.218703 24161 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.195@KRBTEST.COM'} at 127.3.73.195:38421
I20260807 10:53:37.220086 3367 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 10:53:37.241459 24812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:37.225970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36222 (local address 127.3.73.254:35855)
0807 10:53:37.226175 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:37.226180 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:37.226512 (+ 332us) server_negotiation.cc:407] Connection header received
0807 10:53:37.226662 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:37.226666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:37.226734 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:37.226825 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:37.228944 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:37.229735 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:37.231366 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:37.231863 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:37.231993 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:37.235399 (+ 3406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:37.235407 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:37.235412 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:37.235455 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:37.238439 (+ 2984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:37.239247 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:37.239251 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:37.239254 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:37.239395 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:37.239825 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:37.239828 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:37.239830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:37.240175 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:37.240360 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:37.240935 (+ 575us) server_negotiation.cc:299] Negotiation successful
0807 10:53:37.241163 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 10:53:37.246732 24161 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36222:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 10:53:37.255250 24161 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 10:53:37.295323 24834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:37.271480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58464 (local address 127.3.73.193:44389)
0807 10:53:37.283447 (+ 11967us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:37.283462 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:37.284578 (+ 1116us) server_negotiation.cc:407] Connection header received
0807 10:53:37.284935 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:37.284949 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:37.285361 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:37.285681 (+ 320us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:53:37.288124 (+ 2443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:37.289817 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:37.292174 (+ 2357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:37.293929 (+ 1755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:37.294106 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:37.294390 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:37.294758 (+ 368us) server_negotiation.cc:299] Negotiation successful
0807 10:53:37.294956 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11662,"thread_start_us":11409,"threads_started":1}
I20260807 10:53:37.295336 24828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:37.270619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:44389 (local address 127.0.0.1:58464)
0807 10:53:37.284375 (+ 13756us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:37.284407 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 10:53:37.284677 (+ 270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:53:37.287215 (+ 2538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:53:37.287223 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:53:37.287270 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:53:37.287822 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:37.287838 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:37.289996 (+ 2158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:37.290009 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:37.291946 (+ 1937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:37.291966 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:37.294162 (+ 2196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:37.294169 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:37.294400 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:37.294471 (+ 71us) client_negotiation.cc:769] Sending connection context
0807 10:53:37.294702 (+ 231us) client_negotiation.cc:240] Negotiation successful
0807 10:53:37.294939 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13531,"thread_start_us":443,"threads_started":1}
I20260807 10:53:37.296929 24831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:37.270472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:42781 (local address 127.0.0.1:59754)
0807 10:53:37.271484 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:37.271528 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 10:53:37.271722 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:53:37.287856 (+ 16134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:53:37.287861 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:53:37.287963 (+ 102us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:53:37.288377 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:37.288389 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:37.291228 (+ 2839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:37.291235 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:37.294017 (+ 2782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:37.294036 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:37.295976 (+ 1940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:37.295982 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:37.296164 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:37.296210 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 10:53:37.296284 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 10:53:37.296428 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":754,"mutex_wait_us":93,"spinlock_wait_cycles":105728,"thread_start_us":14360,"threads_started":1}
I20260807 10:53:37.297711 24835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:37.271928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:32827 (local address 127.0.0.1:58568)
0807 10:53:37.287263 (+ 15335us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:37.287282 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 10:53:37.287968 (+ 686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:53:37.288763 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:53:37.288869 (+ 106us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:53:37.288903 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:53:37.289400 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:37.289418 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:37.291311 (+ 1893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:37.291316 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:37.293745 (+ 2429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:37.293762 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:37.296240 (+ 2478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:37.296247 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:37.296427 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:37.296454 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 10:53:37.296532 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 10:53:37.296667 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15185,"mutex_wait_us":83,"spinlock_wait_cycles":24192,"thread_start_us":15537,"threads_started":1}
I20260807 10:53:37.297736 24832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:37.270734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59754 (local address 127.3.73.195:42781)
0807 10:53:37.287180 (+ 16446us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:37.287188 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:37.287220 (+ 32us) server_negotiation.cc:407] Connection header received
0807 10:53:37.287322 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:37.287332 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:37.287613 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:37.287852 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:53:37.289228 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:37.291037 (+ 1809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:37.294227 (+ 3190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:37.295713 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:37.296860 (+ 1147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:37.297108 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:37.297184 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 10:53:37.297369 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16261,"thread_start_us":16069,"threads_started":1}
I20260807 10:53:37.299432 24836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:37.272316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58568 (local address 127.3.73.194:32827)
0807 10:53:37.286928 (+ 14612us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:37.286941 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:37.287415 (+ 474us) server_negotiation.cc:407] Connection header received
0807 10:53:37.288241 (+ 826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:37.288253 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:37.288526 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:37.288858 (+ 332us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:53:37.289643 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:37.291164 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:37.293948 (+ 2784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:37.295739 (+ 1791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:37.295876 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:37.296096 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:37.299019 (+ 2923us) server_negotiation.cc:299] Negotiation successful
0807 10:53:37.299173 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14356,"thread_start_us":14642,"threads_started":1}
I20260807 10:53:37.299960 24331 tablet_service.cc:1467] Processing CreateTablet for tablet a4b025a94a8f4d4a9aba2f958ff4ca2e (DEFAULT_TABLE table=test-table [id=e1382607d1a24e4c857ba634a9a81765]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:53:37.301729 24331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4b025a94a8f4d4a9aba2f958ff4ca2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:53:37.301770 24711 tablet_service.cc:1467] Processing CreateTablet for tablet a4b025a94a8f4d4a9aba2f958ff4ca2e (DEFAULT_TABLE table=test-table [id=e1382607d1a24e4c857ba634a9a81765]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:53:37.302568 24519 tablet_service.cc:1467] Processing CreateTablet for tablet a4b025a94a8f4d4a9aba2f958ff4ca2e (DEFAULT_TABLE table=test-table [id=e1382607d1a24e4c857ba634a9a81765]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:53:37.303455 24711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4b025a94a8f4d4a9aba2f958ff4ca2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:53:37.303969 24519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4b025a94a8f4d4a9aba2f958ff4ca2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:53:37.318665 24837 tablet_bootstrap.cc:492] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 2ad1efccd76440a6b62e006eb4a5f83e: Bootstrap starting.
I20260807 10:53:37.319398 24840 tablet_bootstrap.cc:492] T a4b025a94a8f4d4a9aba2f958ff4ca2e P be67427905c64a048b6def3957fdb418: Bootstrap starting.
I20260807 10:53:37.323417 24837 tablet_bootstrap.cc:654] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 2ad1efccd76440a6b62e006eb4a5f83e: Neither blocks nor log segments found. Creating new log.
I20260807 10:53:37.323408 24840 tablet_bootstrap.cc:654] T a4b025a94a8f4d4a9aba2f958ff4ca2e P be67427905c64a048b6def3957fdb418: Neither blocks nor log segments found. Creating new log.
I20260807 10:53:37.324029 24839 tablet_bootstrap.cc:492] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 3425beb307af44598102bae102debd71: Bootstrap starting.
I20260807 10:53:37.324864 24840 log.cc:824] T a4b025a94a8f4d4a9aba2f958ff4ca2e P be67427905c64a048b6def3957fdb418: Log is configured to *not* fsync() on all Append() calls
I20260807 10:53:37.324864 24837 log.cc:824] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 2ad1efccd76440a6b62e006eb4a5f83e: Log is configured to *not* fsync() on all Append() calls
I20260807 10:53:37.327970 24839 tablet_bootstrap.cc:654] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 3425beb307af44598102bae102debd71: Neither blocks nor log segments found. Creating new log.
I20260807 10:53:37.329424 24840 tablet_bootstrap.cc:492] T a4b025a94a8f4d4a9aba2f958ff4ca2e P be67427905c64a048b6def3957fdb418: No bootstrap required, opened a new log
I20260807 10:53:37.329411 24839 log.cc:824] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 3425beb307af44598102bae102debd71: Log is configured to *not* fsync() on all Append() calls
I20260807 10:53:37.329559 24837 tablet_bootstrap.cc:492] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 2ad1efccd76440a6b62e006eb4a5f83e: No bootstrap required, opened a new log
I20260807 10:53:37.329754 24840 ts_tablet_manager.cc:1397] T a4b025a94a8f4d4a9aba2f958ff4ca2e P be67427905c64a048b6def3957fdb418: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260807 10:53:37.329819 24837 ts_tablet_manager.cc:1397] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 2ad1efccd76440a6b62e006eb4a5f83e: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.006s
I20260807 10:53:37.332312 24839 tablet_bootstrap.cc:492] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 3425beb307af44598102bae102debd71: No bootstrap required, opened a new log
I20260807 10:53:37.332638 24839 ts_tablet_manager.cc:1397] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 3425beb307af44598102bae102debd71: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260807 10:53:37.341032 24839 raft_consensus.cc:348] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 3425beb307af44598102bae102debd71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be67427905c64a048b6def3957fdb418" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 42781 } } peers { permanent_uuid: "3425beb307af44598102bae102debd71" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44389 } } peers { permanent_uuid: "2ad1efccd76440a6b62e006eb4a5f83e" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 32827 } }
I20260807 10:53:37.341501 24839 raft_consensus.cc:374] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 3425beb307af44598102bae102debd71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:53:37.341287 24840 raft_consensus.cc:348] T a4b025a94a8f4d4a9aba2f958ff4ca2e P be67427905c64a048b6def3957fdb418 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be67427905c64a048b6def3957fdb418" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 42781 } } peers { permanent_uuid: "3425beb307af44598102bae102debd71" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44389 } } peers { permanent_uuid: "2ad1efccd76440a6b62e006eb4a5f83e" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 32827 } }
I20260807 10:53:37.341599 24839 raft_consensus.cc:729] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 3425beb307af44598102bae102debd71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3425beb307af44598102bae102debd71, State: Initialized, Role: FOLLOWER
I20260807 10:53:37.341765 24840 raft_consensus.cc:374] T a4b025a94a8f4d4a9aba2f958ff4ca2e P be67427905c64a048b6def3957fdb418 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:53:37.341888 24840 raft_consensus.cc:729] T a4b025a94a8f4d4a9aba2f958ff4ca2e P be67427905c64a048b6def3957fdb418 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be67427905c64a048b6def3957fdb418, State: Initialized, Role: FOLLOWER
I20260807 10:53:37.342370 24839 consensus_queue.cc:260] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 3425beb307af44598102bae102debd71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be67427905c64a048b6def3957fdb418" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 42781 } } peers { permanent_uuid: "3425beb307af44598102bae102debd71" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44389 } } peers { permanent_uuid: "2ad1efccd76440a6b62e006eb4a5f83e" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 32827 } }
I20260807 10:53:37.342350 24837 raft_consensus.cc:348] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 2ad1efccd76440a6b62e006eb4a5f83e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be67427905c64a048b6def3957fdb418" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 42781 } } peers { permanent_uuid: "3425beb307af44598102bae102debd71" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44389 } } peers { permanent_uuid: "2ad1efccd76440a6b62e006eb4a5f83e" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 32827 } }
I20260807 10:53:37.342527 24840 consensus_queue.cc:260] T a4b025a94a8f4d4a9aba2f958ff4ca2e P be67427905c64a048b6def3957fdb418 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be67427905c64a048b6def3957fdb418" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 42781 } } peers { permanent_uuid: "3425beb307af44598102bae102debd71" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44389 } } peers { permanent_uuid: "2ad1efccd76440a6b62e006eb4a5f83e" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 32827 } }
I20260807 10:53:37.342859 24837 raft_consensus.cc:374] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 2ad1efccd76440a6b62e006eb4a5f83e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:53:37.343112 24837 raft_consensus.cc:729] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 2ad1efccd76440a6b62e006eb4a5f83e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ad1efccd76440a6b62e006eb4a5f83e, State: Initialized, Role: FOLLOWER
I20260807 10:53:37.344134 24808 heartbeater.cc:503] Master 127.3.73.254:35855 was elected leader, sending a full tablet report...
I20260807 10:53:37.344056 24837 consensus_queue.cc:260] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 2ad1efccd76440a6b62e006eb4a5f83e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be67427905c64a048b6def3957fdb418" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 42781 } } peers { permanent_uuid: "3425beb307af44598102bae102debd71" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44389 } } peers { permanent_uuid: "2ad1efccd76440a6b62e006eb4a5f83e" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 32827 } }
I20260807 10:53:37.345465 24614 heartbeater.cc:503] Master 127.3.73.254:35855 was elected leader, sending a full tablet report...
I20260807 10:53:37.346575 24840 ts_tablet_manager.cc:1428] T a4b025a94a8f4d4a9aba2f958ff4ca2e P be67427905c64a048b6def3957fdb418: Time spent starting tablet: real 0.017s user 0.010s sys 0.005s
I20260807 10:53:37.349153 24837 ts_tablet_manager.cc:1428] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 2ad1efccd76440a6b62e006eb4a5f83e: Time spent starting tablet: real 0.019s user 0.011s sys 0.006s
I20260807 10:53:37.353631 24839 ts_tablet_manager.cc:1428] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 3425beb307af44598102bae102debd71: Time spent starting tablet: real 0.021s user 0.013s sys 0.004s
I20260807 10:53:37.428203 24844 raft_consensus.cc:482] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 3425beb307af44598102bae102debd71 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 10:53:37.428860 24844 raft_consensus.cc:504] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 3425beb307af44598102bae102debd71 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be67427905c64a048b6def3957fdb418" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 42781 } } peers { permanent_uuid: "3425beb307af44598102bae102debd71" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44389 } } peers { permanent_uuid: "2ad1efccd76440a6b62e006eb4a5f83e" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 32827 } }
I20260807 10:53:37.431074 24844 leader_election.cc:288] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 3425beb307af44598102bae102debd71 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be67427905c64a048b6def3957fdb418 (127.3.73.195:42781), 2ad1efccd76440a6b62e006eb4a5f83e (127.3.73.194:32827)
W20260807 10:53:37.436522 24810 tablet.cc:2369] T a4b025a94a8f4d4a9aba2f958ff4ca2e P be67427905c64a048b6def3957fdb418: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:53:37.439584 24832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:37.431551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:48569 (local address 127.3.73.195:42781)
0807 10:53:37.431831 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:37.431841 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:37.433027 (+ 1186us) server_negotiation.cc:407] Connection header received
0807 10:53:37.433346 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:37.433350 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:37.433449 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:37.433563 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:53:37.434417 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:37.435398 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:37.437775 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:37.438561 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:37.438770 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:37.438857 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:37.439173 (+ 316us) server_negotiation.cc:299] Negotiation successful
0807 10:53:37.439319 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260807 10:53:37.439577 24836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:37.431682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:51901 (local address 127.3.73.194:32827)
0807 10:53:37.431943 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:37.431953 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:37.433024 (+ 1071us) server_negotiation.cc:407] Connection header received
0807 10:53:37.433272 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:37.433279 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:37.433387 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:37.433511 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:53:37.434210 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:37.435254 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:37.437456 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:37.438354 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:37.438567 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:37.438659 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:37.439095 (+ 436us) server_negotiation.cc:299] Negotiation successful
0807 10:53:37.439263 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 10:53:37.439702 24850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:37.431416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:32827 (local address 127.3.73.193:51901)
0807 10:53:37.432877 (+ 1461us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:37.432897 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 10:53:37.433089 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:53:37.433559 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:53:37.433563 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:53:37.433588 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:53:37.434022 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:37.434032 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:37.435397 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:37.435404 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:37.437270 (+ 1866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:37.437286 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:37.438612 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:37.438618 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:37.438897 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:37.438926 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 10:53:37.439049 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 10:53:37.439211 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1354,"mutex_wait_us":42,"thread_start_us":871,"threads_started":1}
I20260807 10:53:37.439754 24849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:37.431300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:42781 (local address 127.3.73.193:48569)
0807 10:53:37.432865 (+ 1565us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:37.432886 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 10:53:37.433170 (+ 284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:53:37.433625 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:53:37.433630 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:53:37.433664 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:53:37.434245 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:37.434259 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:37.435559 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:37.435564 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:37.437599 (+ 2035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:37.437612 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:37.438798 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:37.438805 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:37.439012 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:37.439035 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 10:53:37.439142 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 10:53:37.439290 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1331,"mutex_wait_us":57,"spinlock_wait_cycles":31104,"thread_start_us":836,"threads_started":1}
I20260807 10:53:37.441170 24737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4b025a94a8f4d4a9aba2f958ff4ca2e" candidate_uuid: "3425beb307af44598102bae102debd71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be67427905c64a048b6def3957fdb418" is_pre_election: true
I20260807 10:53:37.441636 24551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4b025a94a8f4d4a9aba2f958ff4ca2e" candidate_uuid: "3425beb307af44598102bae102debd71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ad1efccd76440a6b62e006eb4a5f83e" is_pre_election: true
I20260807 10:53:37.441741 24737 raft_consensus.cc:2454] T a4b025a94a8f4d4a9aba2f958ff4ca2e P be67427905c64a048b6def3957fdb418 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3425beb307af44598102bae102debd71 in term 0.
I20260807 10:53:37.442168 24551 raft_consensus.cc:2454] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 2ad1efccd76440a6b62e006eb4a5f83e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3425beb307af44598102bae102debd71 in term 0.
I20260807 10:53:37.442601 24267 leader_election.cc:302] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 3425beb307af44598102bae102debd71 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3425beb307af44598102bae102debd71, be67427905c64a048b6def3957fdb418; no voters:
I20260807 10:53:37.443145 24844 raft_consensus.cc:2781] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 3425beb307af44598102bae102debd71 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 10:53:37.443311 24844 raft_consensus.cc:482] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 3425beb307af44598102bae102debd71 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 10:53:37.443486 24844 raft_consensus.cc:3037] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 3425beb307af44598102bae102debd71 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:53:37.445936 24844 raft_consensus.cc:504] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 3425beb307af44598102bae102debd71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be67427905c64a048b6def3957fdb418" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 42781 } } peers { permanent_uuid: "3425beb307af44598102bae102debd71" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44389 } } peers { permanent_uuid: "2ad1efccd76440a6b62e006eb4a5f83e" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 32827 } }
I20260807 10:53:37.446694 24844 leader_election.cc:288] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 3425beb307af44598102bae102debd71 [CANDIDATE]: Term 1 election: Requested vote from peers be67427905c64a048b6def3957fdb418 (127.3.73.195:42781), 2ad1efccd76440a6b62e006eb4a5f83e (127.3.73.194:32827)
I20260807 10:53:37.447227 24737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4b025a94a8f4d4a9aba2f958ff4ca2e" candidate_uuid: "3425beb307af44598102bae102debd71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be67427905c64a048b6def3957fdb418"
I20260807 10:53:37.447487 24737 raft_consensus.cc:3037] T a4b025a94a8f4d4a9aba2f958ff4ca2e P be67427905c64a048b6def3957fdb418 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:53:37.447435 24551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4b025a94a8f4d4a9aba2f958ff4ca2e" candidate_uuid: "3425beb307af44598102bae102debd71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ad1efccd76440a6b62e006eb4a5f83e"
I20260807 10:53:37.447700 24551 raft_consensus.cc:3037] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 2ad1efccd76440a6b62e006eb4a5f83e [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:53:37.449983 24737 raft_consensus.cc:2454] T a4b025a94a8f4d4a9aba2f958ff4ca2e P be67427905c64a048b6def3957fdb418 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3425beb307af44598102bae102debd71 in term 1.
I20260807 10:53:37.450502 24267 leader_election.cc:302] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 3425beb307af44598102bae102debd71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3425beb307af44598102bae102debd71, be67427905c64a048b6def3957fdb418; no voters:
I20260807 10:53:37.450567 24551 raft_consensus.cc:2454] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 2ad1efccd76440a6b62e006eb4a5f83e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3425beb307af44598102bae102debd71 in term 1.
I20260807 10:53:37.450862 24844 raft_consensus.cc:2781] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 3425beb307af44598102bae102debd71 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:53:37.451498 24844 raft_consensus.cc:686] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 3425beb307af44598102bae102debd71 [term 1 LEADER]: Becoming Leader. State: Replica: 3425beb307af44598102bae102debd71, State: Running, Role: LEADER
I20260807 10:53:37.452100 24844 consensus_queue.cc:237] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 3425beb307af44598102bae102debd71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be67427905c64a048b6def3957fdb418" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 42781 } } peers { permanent_uuid: "3425beb307af44598102bae102debd71" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44389 } } peers { permanent_uuid: "2ad1efccd76440a6b62e006eb4a5f83e" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 32827 } }
I20260807 10:53:37.457432 24161 catalog_manager.cc:5495] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 3425beb307af44598102bae102debd71 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3425beb307af44598102bae102debd71 (127.3.73.193). New cstate: current_term: 1 leader_uuid: "3425beb307af44598102bae102debd71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be67427905c64a048b6def3957fdb418" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 42781 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3425beb307af44598102bae102debd71" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ad1efccd76440a6b62e006eb4a5f83e" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 32827 } health_report { overall_health: UNKNOWN } } }
W20260807 10:53:37.485136 24427 tablet.cc:2369] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 3425beb307af44598102bae102debd71: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:53:37.495906 24834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:37.489478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58468 (local address 127.3.73.193:44389)
0807 10:53:37.489685 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:37.489689 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:37.489828 (+ 139us) server_negotiation.cc:407] Connection header received
0807 10:53:37.490193 (+ 365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:37.490200 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:37.490313 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:37.490475 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:53:37.490998 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:37.491774 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:37.493556 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:37.493987 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:37.494093 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:37.494764 (+ 671us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:53:37.495319 (+ 555us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:53:37.495442 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:37.495605 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 10:53:37.495683 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 10:53:37.506215 24737 raft_consensus.cc:1261] T a4b025a94a8f4d4a9aba2f958ff4ca2e P be67427905c64a048b6def3957fdb418 [term 1 FOLLOWER]: Refusing update from remote peer 3425beb307af44598102bae102debd71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:53:37.506253 24551 raft_consensus.cc:1261] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 2ad1efccd76440a6b62e006eb4a5f83e [term 1 FOLLOWER]: Refusing update from remote peer 3425beb307af44598102bae102debd71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:53:37.507601 24851 consensus_queue.cc:1035] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 3425beb307af44598102bae102debd71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ad1efccd76440a6b62e006eb4a5f83e" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 32827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:53:37.508066 24844 consensus_queue.cc:1035] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 3425beb307af44598102bae102debd71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be67427905c64a048b6def3957fdb418" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 42781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:53:37.523279 24855 mvcc.cc:205] Tried to move back new op lower bound from 7315865671692587008 to 7315865671488720896. Current Snapshot: MvccSnapshot[applied={T|T < 7315865671692587008}]
I20260807 10:53:37.526834 24856 mvcc.cc:205] Tried to move back new op lower bound from 7315865671692587008 to 7315865671488720896. Current Snapshot: MvccSnapshot[applied={T|T < 7315865671692587008}]
I20260807 10:53:37.542788 24161 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36222:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:53:37.543424 24161 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36222:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:53:37.548247 24161 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05: Sending DeleteTablet for 3 replicas of tablet a4b025a94a8f4d4a9aba2f958ff4ca2e
I20260807 10:53:37.549795 24711 tablet_service.cc:1514] Processing DeleteTablet for tablet a4b025a94a8f4d4a9aba2f958ff4ca2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:53:37 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:59754
I20260807 10:53:37.549845 24331 tablet_service.cc:1514] Processing DeleteTablet for tablet a4b025a94a8f4d4a9aba2f958ff4ca2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:53:37 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:58464
I20260807 10:53:37.550254 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 24246
I20260807 10:53:37.551059 24519 tablet_service.cc:1514] Processing DeleteTablet for tablet a4b025a94a8f4d4a9aba2f958ff4ca2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:53:37 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:58568
I20260807 10:53:37.561136 24878 tablet_replica.cc:330] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 2ad1efccd76440a6b62e006eb4a5f83e: stopping tablet replica
I20260807 10:53:37.561726 24878 raft_consensus.cc:2229] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 2ad1efccd76440a6b62e006eb4a5f83e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:53:37.562391 24878 raft_consensus.cc:2258] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 2ad1efccd76440a6b62e006eb4a5f83e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:53:37.567616 24875 tablet_replica.cc:330] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 3425beb307af44598102bae102debd71: stopping tablet replica
I20260807 10:53:37.567759 24878 ts_tablet_manager.cc:1905] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 2ad1efccd76440a6b62e006eb4a5f83e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:53:37.568642 24875 raft_consensus.cc:2229] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 3425beb307af44598102bae102debd71 [term 1 LEADER]: Raft consensus shutting down.
I20260807 10:53:37.569846 24875 raft_consensus.cc:2258] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 3425beb307af44598102bae102debd71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:53:37.570987 24876 tablet_replica.cc:330] T a4b025a94a8f4d4a9aba2f958ff4ca2e P be67427905c64a048b6def3957fdb418: stopping tablet replica
I20260807 10:53:37.571446 24876 raft_consensus.cc:2229] T a4b025a94a8f4d4a9aba2f958ff4ca2e P be67427905c64a048b6def3957fdb418 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:53:37.571967 24876 raft_consensus.cc:2258] T a4b025a94a8f4d4a9aba2f958ff4ca2e P be67427905c64a048b6def3957fdb418 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:53:37.574283 24875 ts_tablet_manager.cc:1905] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 3425beb307af44598102bae102debd71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:53:37.580832 24878 ts_tablet_manager.cc:1918] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 2ad1efccd76440a6b62e006eb4a5f83e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:53:37.581151 24878 log.cc:1196] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 2ad1efccd76440a6b62e006eb4a5f83e: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/wal/wals/a4b025a94a8f4d4a9aba2f958ff4ca2e
I20260807 10:53:37.581822 24878 ts_tablet_manager.cc:1939] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 2ad1efccd76440a6b62e006eb4a5f83e: Deleting consensus metadata
I20260807 10:53:37.582442 24875 ts_tablet_manager.cc:1918] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 3425beb307af44598102bae102debd71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:53:37.582647 24875 log.cc:1196] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 3425beb307af44598102bae102debd71: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/wal/wals/a4b025a94a8f4d4a9aba2f958ff4ca2e
I20260807 10:53:37.583344 24875 ts_tablet_manager.cc:1939] T a4b025a94a8f4d4a9aba2f958ff4ca2e P 3425beb307af44598102bae102debd71: Deleting consensus metadata
I20260807 10:53:37.584215 24141 catalog_manager.cc:4841] TS 2ad1efccd76440a6b62e006eb4a5f83e (127.3.73.194:32827): tablet a4b025a94a8f4d4a9aba2f958ff4ca2e (table test-table [id=e1382607d1a24e4c857ba634a9a81765]) successfully deleted
I20260807 10:53:37.587064 24876 ts_tablet_manager.cc:1905] T a4b025a94a8f4d4a9aba2f958ff4ca2e P be67427905c64a048b6def3957fdb418: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 10:53:37 dist-test-slave-x5fp krb5kdc[24097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100014, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.193@KRBTEST.COM
I20260807 10:53:37.595960 24876 ts_tablet_manager.cc:1918] T a4b025a94a8f4d4a9aba2f958ff4ca2e P be67427905c64a048b6def3957fdb418: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:53:37.596220 24876 log.cc:1196] T a4b025a94a8f4d4a9aba2f958ff4ca2e P be67427905c64a048b6def3957fdb418: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/wal/wals/a4b025a94a8f4d4a9aba2f958ff4ca2e
I20260807 10:53:37.597623 24141 catalog_manager.cc:4841] TS 3425beb307af44598102bae102debd71 (127.3.73.193:44389): tablet a4b025a94a8f4d4a9aba2f958ff4ca2e (table test-table [id=e1382607d1a24e4c857ba634a9a81765]) successfully deleted
I20260807 10:53:37.598372 24876 ts_tablet_manager.cc:1939] T a4b025a94a8f4d4a9aba2f958ff4ca2e P be67427905c64a048b6def3957fdb418: Deleting consensus metadata
I20260807 10:53:37.601039 24139 catalog_manager.cc:4841] TS be67427905c64a048b6def3957fdb418 (127.3.73.195:42781): tablet a4b025a94a8f4d4a9aba2f958ff4ca2e (table test-table [id=e1382607d1a24e4c857ba634a9a81765]) successfully deleted
I20260807 10:53:37.603513 24834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:37.551613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58484 (local address 127.3.73.193:44389)
0807 10:53:37.551836 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:37.551843 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:37.574050 (+ 22207us) server_negotiation.cc:407] Connection header received
0807 10:53:37.574297 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:37.574305 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:37.574421 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:37.574566 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:37.581410 (+ 6844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:37.582828 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:37.585593 (+ 2765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:37.586444 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:37.586599 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:37.593399 (+ 6800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:37.593422 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:37.593444 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:37.593509 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:37.598575 (+ 5066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:37.600052 (+ 1477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:37.600065 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:37.600069 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:37.600310 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:37.600948 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:37.600954 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:37.600957 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:37.601492 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:37.601896 (+ 404us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:37.602991 (+ 1095us) server_negotiation.cc:299] Negotiation successful
0807 10:53:37.603209 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 10:53:37.604395 24419 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:53:37.712627 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 24246
I20260807 10:53:37.744626 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 24433
Aug 07 10:53:37 dist-test-slave-x5fp krb5kdc[24097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100014, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.194@KRBTEST.COM
I20260807 10:53:37.767931 24836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:37.745669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58572 (local address 127.3.73.194:32827)
0807 10:53:37.745993 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:37.746000 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:37.746024 (+ 24us) server_negotiation.cc:407] Connection header received
0807 10:53:37.746233 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:37.746238 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:37.746330 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:37.746467 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:37.747984 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:37.748935 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:37.750447 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:37.750904 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:37.751057 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:37.761404 (+ 10347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:37.761426 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:37.761444 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:37.761491 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:37.764960 (+ 3469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:37.765781 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:37.765789 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:37.765791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:37.765929 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:37.766318 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:37.766324 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:37.766327 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:37.766709 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:37.767067 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:37.767335 (+ 268us) server_negotiation.cc:299] Negotiation successful
0807 10:53:37.767571 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171}
I20260807 10:53:37.768796 24607 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:53:37.863129 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 24433
I20260807 10:53:37.890266 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 24623
Aug 07 10:53:37 dist-test-slave-x5fp krb5kdc[24097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100014, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.195@KRBTEST.COM
I20260807 10:53:37.909694 24832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:37.891162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59758 (local address 127.3.73.195:42781)
0807 10:53:37.891339 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:37.891344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:37.891413 (+ 69us) server_negotiation.cc:407] Connection header received
0807 10:53:37.891575 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:37.891578 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:37.891649 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:37.891743 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:37.893461 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:37.894190 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:37.895656 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:37.896195 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:37.896324 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:37.902062 (+ 5738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:37.902083 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:37.902102 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:37.902160 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:37.906036 (+ 3876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:37.907226 (+ 1190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:37.907238 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:37.907242 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:37.907449 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:37.908043 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:37.908047 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:37.908049 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:37.908482 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:37.908871 (+ 389us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:37.909209 (+ 338us) server_negotiation.cc:299] Negotiation successful
0807 10:53:37.909404 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 10:53:37.910527 24801 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:53:38.011684 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 24623
I20260807 10:53:38.040555 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 24115
I20260807 10:53:38.042320 24209 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:53:38.176594 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 24115
I20260807 10:53:38.211169 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:35855
--webserver_interface=127.3.73.254
--webserver_port=39523
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:46781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:35855
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:53:38.599823 24884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:53:38.600168 24884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:53:38.600272 24884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:53:38.607856 24884 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:53:38.607995 24884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:53:38.608044 24884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:53:38.608096 24884 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:53:38.608143 24884 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:53:38.618541 24884 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:46781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:35855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:35855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=39523
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:53:38.619906 24884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:53:38.621402 24884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:53:38.650789 24903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:38.650839 24901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:38.650749 24899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:53:38.654633 24884 server_base.cc:1034] running on GCE node
I20260807 10:53:38.655731 24884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:53:38.657757 24884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:53:38.659044 24884 hybrid_clock.cc:648] HybridClock initialized: now 1786100018658987 us; error 64 us; skew 500 ppm
I20260807 10:53:38.659544 24884 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:53:38 dist-test-slave-x5fp krb5kdc[24097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100018, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:53:38.663861 24884 init.cc:377] Logged in from keytab as kudu/127.3.73.254@KRBTEST.COM (short username kudu)
I20260807 10:53:38.665972 24884 webserver.cc:460] Webserver started at http://127.3.73.254:39523/ using document root <none> and password file <none>
I20260807 10:53:38.666918 24884 fs_manager.cc:359] Metadata directory not provided
I20260807 10:53:38.667096 24884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:53:38.674079 24884 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 10:53:38.677722 24913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:38.679410 24884 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 10:53:38.679620 24884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "e9c8d1c8814f4b03b923ff6cdc695c05"
format_stamp: "Formatted at 2026-08-07 10:53:35 on dist-test-slave-x5fp"
server_key: "4a762993e8207c6ceee90645f7869f5c"
server_key_iv: "8601664132f7a6639e84a2aaa65819e7"
server_key_version: "encryptionkey@0"
I20260807 10:53:38.681002 24884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:53:38.716003 24884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:53:38.721503 24884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:53:38.721853 24884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:53:38.745397 24884 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:35855
I20260807 10:53:38.745414 24994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:35855 every 8 connection(s)
I20260807 10:53:38.747272 24884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:53:38.748486 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 24884
I20260807 10:53:38.749481 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.193:44389
--local_ip_for_outbound_sockets=127.3.73.193
--tserver_master_addrs=127.3.73.254:35855
--webserver_port=45273
--webserver_interface=127.3.73.193
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:46781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 10:53:38.756031 24996 sys_catalog.cc:263] Verifying existing consensus state
I20260807 10:53:38.760118 24996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05: Bootstrap starting.
I20260807 10:53:38.782975 24996 log.cc:824] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05: Log is configured to *not* fsync() on all Append() calls
I20260807 10:53:38.797677 24996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 10:53:38.798341 24996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05: Bootstrap complete.
I20260807 10:53:38.809139 24996 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9c8d1c8814f4b03b923ff6cdc695c05" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 35855 } }
I20260807 10:53:38.810170 24996 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9c8d1c8814f4b03b923ff6cdc695c05, State: Initialized, Role: FOLLOWER
I20260807 10:53:38.810951 24996 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9c8d1c8814f4b03b923ff6cdc695c05" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 35855 } }
I20260807 10:53:38.811249 24996 raft_consensus.cc:388] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:53:38.811414 24996 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:53:38.811614 24996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [term 1 FOLLOWER]: Advancing to term 2
I20260807 10:53:38.816306 24996 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9c8d1c8814f4b03b923ff6cdc695c05" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 35855 } }
I20260807 10:53:38.817058 24996 leader_election.cc:302] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9c8d1c8814f4b03b923ff6cdc695c05; no voters:
I20260807 10:53:38.817824 24996 leader_election.cc:288] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 10:53:38.818001 25003 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 10:53:38.819464 25003 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [term 2 LEADER]: Becoming Leader. State: Replica: e9c8d1c8814f4b03b923ff6cdc695c05, State: Running, Role: LEADER
I20260807 10:53:38.820132 25003 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9c8d1c8814f4b03b923ff6cdc695c05" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 35855 } }
I20260807 10:53:38.820704 24996 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:53:38.823315 25004 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e9c8d1c8814f4b03b923ff6cdc695c05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9c8d1c8814f4b03b923ff6cdc695c05" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 35855 } } }
I20260807 10:53:38.823319 25006 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9c8d1c8814f4b03b923ff6cdc695c05. Latest consensus state: current_term: 2 leader_uuid: "e9c8d1c8814f4b03b923ff6cdc695c05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9c8d1c8814f4b03b923ff6cdc695c05" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 35855 } } }
I20260807 10:53:38.823915 25006 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [sys.catalog]: This master's current role is: LEADER
I20260807 10:53:38.823910 25004 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [sys.catalog]: This master's current role is: LEADER
I20260807 10:53:38.833606 25010 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:53:38.845323 25010 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:53:38.847489 25010 catalog_manager.cc:1259] Loaded cluster ID: e4f39d529d4e4402bd5703345d6a4a30
I20260807 10:53:38.847640 25010 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:53:38.852555 25010 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 10:53:38.878894 25010 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 10:53:38.887610 25010 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:53:38.889271 25010 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05: Loaded TSK: 0
I20260807 10:53:38.890605 25010 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 10:53:39.170279 24999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:53:39.170679 24999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:53:39.170848 24999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:53:39.178828 24999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:53:39.179013 24999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:53:39.179207 24999 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.193
I20260807 10:53:39.190254 24999 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:46781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.193:44389
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.73.193
--webserver_port=45273
--tserver_master_addrs=127.3.73.254:35855
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.193
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:53:39.191738 24999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:53:39.193334 24999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:53:39.209362 25039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:39.209649 25041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:39.210047 25037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:53:39.211086 24999 server_base.cc:1034] running on GCE node
I20260807 10:53:39.211936 24999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:53:39.213313 24999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:53:39.214584 24999 hybrid_clock.cc:648] HybridClock initialized: now 1786100019214504 us; error 64 us; skew 500 ppm
I20260807 10:53:39.214996 24999 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:53:39 dist-test-slave-x5fp krb5kdc[24097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100019, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:53:39.219269 24999 init.cc:377] Logged in from keytab as kudu/127.3.73.193@KRBTEST.COM (short username kudu)
I20260807 10:53:39.222644 24999 webserver.cc:460] Webserver started at http://127.3.73.193:45273/ using document root <none> and password file <none>
I20260807 10:53:39.223657 24999 fs_manager.cc:359] Metadata directory not provided
I20260807 10:53:39.223845 24999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:53:39.231031 24999 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 10:53:39.234979 25055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:39.237102 24999 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 10:53:39.237507 24999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/wal
uuid: "3425beb307af44598102bae102debd71"
format_stamp: "Formatted at 2026-08-07 10:53:35 on dist-test-slave-x5fp"
server_key: "7d2261ea05a9dba3968af1cd990cb68d"
server_key_iv: "57de6d69b5f35dacf8f12537c6805b32"
server_key_version: "encryptionkey@0"
I20260807 10:53:39.238716 24999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:53:39.268854 24999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:53:39.273900 24999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:53:39.274212 24999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:53:39.275489 24999 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:53:39.277160 24999 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:53:39.277254 24999 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:39.277423 24999 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:53:39.277509 24999 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:39.314554 24999 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.193:44389
I20260807 10:53:39.314682 25226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.193:44389 every 8 connection(s)
I20260807 10:53:39.316177 24999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
I20260807 10:53:39.326325 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 24999
I20260807 10:53:39.327047 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.194:32827
--local_ip_for_outbound_sockets=127.3.73.194
--tserver_master_addrs=127.3.73.254:35855
--webserver_port=42997
--webserver_interface=127.3.73.194
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:46781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 10:53:39 dist-test-slave-x5fp krb5kdc[24097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100019, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:53:39.347195 25234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:39.319568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.193:39147)
0807 10:53:39.320484 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:39.320531 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 10:53:39.322902 (+ 2371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:53:39.324545 (+ 1643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:53:39.324558 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:53:39.325335 (+ 777us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:53:39.325985 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:39.326008 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:39.328064 (+ 2056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:39.328073 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:39.330381 (+ 2308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:39.330402 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:39.331664 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:39.331674 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:39.332037 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:39.333013 (+ 976us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:53:39.333051 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:53:39.336287 (+ 3236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:53:39.341537 (+ 5250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:39.341551 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:39.341574 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:39.342359 (+ 785us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:39.343183 (+ 824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:39.343187 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:39.343190 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:39.343463 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:39.344864 (+ 1401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:53:39.344871 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:53:39.345256 (+ 385us) client_negotiation.cc:769] Sending connection context
0807 10:53:39.345547 (+ 291us) client_negotiation.cc:240] Negotiation successful
0807 10:53:39.346033 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":372,"threads_started":1}
I20260807 10:53:39.348446 25230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:39.319281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:39147 (local address 127.3.73.254:35855)
0807 10:53:39.320047 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:39.320069 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:39.320701 (+ 632us) server_negotiation.cc:407] Connection header received
0807 10:53:39.323559 (+ 2858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:39.323573 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:39.323967 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:39.324370 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:39.326249 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:39.327823 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:39.330601 (+ 2778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:39.331378 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:39.331547 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:39.336687 (+ 5140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:39.336698 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:39.336769 (+ 71us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:39.336838 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:39.341279 (+ 4441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:39.342669 (+ 1390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:39.342679 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:39.342687 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:39.343025 (+ 338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:39.343615 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:39.343624 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:39.343631 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:39.344582 (+ 951us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:39.344876 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:39.345937 (+ 1061us) server_negotiation.cc:299] Negotiation successful
0807 10:53:39.346602 (+ 665us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":283,"threads_started":1}
I20260807 10:53:39.351899 25227 heartbeater.cc:347] Connected to a master server at 127.3.73.254:35855
I20260807 10:53:39.352362 25227 heartbeater.cc:464] Registering TS with master...
I20260807 10:53:39.353473 25227 heartbeater.cc:511] Master 127.3.73.254:35855 requested a full tablet report, sending...
I20260807 10:53:39.356441 24943 ts_manager.cc:194] Registered new tserver with Master: 3425beb307af44598102bae102debd71 (127.3.73.193:44389)
I20260807 10:53:39.358343 24943 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.193@KRBTEST.COM'} at 127.3.73.193:39147
W20260807 10:53:39.713630 25235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:53:39.713979 25235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:53:39.714148 25235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:53:39.722591 25235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:53:39.722736 25235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:53:39.722972 25235 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.194
I20260807 10:53:39.733687 25235 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:46781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.194:32827
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.73.194
--webserver_port=42997
--tserver_master_addrs=127.3.73.254:35855
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.194
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:53:39.735447 25235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:53:39.737150 25235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:53:39.752784 25250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:39.752878 25248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:39.753062 25245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:53:39.759148 25235 server_base.cc:1034] running on GCE node
I20260807 10:53:39.759990 25235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:53:39.761432 25235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:53:39.762698 25235 hybrid_clock.cc:648] HybridClock initialized: now 1786100019762615 us; error 69 us; skew 500 ppm
I20260807 10:53:39.763085 25235 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:53:39 dist-test-slave-x5fp krb5kdc[24097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100019, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:53:39.767093 25235 init.cc:377] Logged in from keytab as kudu/127.3.73.194@KRBTEST.COM (short username kudu)
I20260807 10:53:39.769177 25235 webserver.cc:460] Webserver started at http://127.3.73.194:42997/ using document root <none> and password file <none>
I20260807 10:53:39.770021 25235 fs_manager.cc:359] Metadata directory not provided
I20260807 10:53:39.770207 25235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:53:39.777060 25235 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 10:53:39.780272 25259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:39.782018 25235 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 10:53:39.782248 25235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/wal
uuid: "2ad1efccd76440a6b62e006eb4a5f83e"
format_stamp: "Formatted at 2026-08-07 10:53:36 on dist-test-slave-x5fp"
server_key: "a578eea0239e45fb35269d810a4448bc"
server_key_iv: "941c49b2167c89b10e5f364a357476b9"
server_key_version: "encryptionkey@0"
I20260807 10:53:39.783437 25235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:53:39.799998 25235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:53:39.805553 25235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:53:39.805956 25235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:53:39.807137 25235 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:53:39.808774 25235 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:53:39.808951 25235 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260807 10:53:39.809104 25235 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:53:39.809198 25235 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:39.848606 25235 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.194:32827
I20260807 10:53:39.848665 25435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.194:32827 every 8 connection(s)
I20260807 10:53:39.850437 25235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
I20260807 10:53:39.859930 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 25235
I20260807 10:53:39.860623 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.195:42781
--local_ip_for_outbound_sockets=127.3.73.195
--tserver_master_addrs=127.3.73.254:35855
--webserver_port=44503
--webserver_interface=127.3.73.195
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:46781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 10:53:39 dist-test-slave-x5fp krb5kdc[24097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100019, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:53:39.874557 25440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:39.853246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:57053 (local address 127.3.73.254:35855)
0807 10:53:39.853793 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:39.853797 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:39.854823 (+ 1026us) server_negotiation.cc:407] Connection header received
0807 10:53:39.857090 (+ 2267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:39.857096 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:39.857220 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:39.857366 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:39.859171 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:39.859972 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:39.861955 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:39.862448 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:39.862605 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:39.866860 (+ 4255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:39.866869 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:39.866878 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:39.866928 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:39.870850 (+ 3922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:39.871804 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:39.871813 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:39.871819 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:39.872018 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:39.872594 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:39.872602 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:39.872607 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:39.873058 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:39.873231 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:39.874028 (+ 797us) server_negotiation.cc:299] Negotiation successful
0807 10:53:39.874247 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":291,"threads_started":1}
I20260807 10:53:39.876825 25444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:39.853819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.194:57053)
0807 10:53:39.854624 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:39.854667 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 10:53:39.856690 (+ 2023us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:53:39.857663 (+ 973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:53:39.857673 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:53:39.858293 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:53:39.858948 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:39.858969 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:39.860169 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:39.860174 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:39.861778 (+ 1604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:39.861793 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:39.862740 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:39.862749 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:39.863012 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:39.863852 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:53:39.863877 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:53:39.866588 (+ 2711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:53:39.871046 (+ 4458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:39.871055 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:39.871071 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:39.871642 (+ 571us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:39.872181 (+ 539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:39.872185 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:39.872188 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:39.872456 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:39.873207 (+ 751us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:53:39.873214 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:53:39.873605 (+ 391us) client_negotiation.cc:769] Sending connection context
0807 10:53:39.873965 (+ 360us) client_negotiation.cc:240] Negotiation successful
0807 10:53:39.875050 (+ 1085us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":311,"threads_started":1}
I20260807 10:53:39.878799 25436 heartbeater.cc:347] Connected to a master server at 127.3.73.254:35855
I20260807 10:53:39.879190 25436 heartbeater.cc:464] Registering TS with master...
I20260807 10:53:39.880023 25436 heartbeater.cc:511] Master 127.3.73.254:35855 requested a full tablet report, sending...
I20260807 10:53:39.881736 24943 ts_manager.cc:194] Registered new tserver with Master: 2ad1efccd76440a6b62e006eb4a5f83e (127.3.73.194:32827)
I20260807 10:53:39.882622 24943 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.194@KRBTEST.COM'} at 127.3.73.194:57053
W20260807 10:53:40.247287 25445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:53:40.247674 25445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:53:40.247864 25445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:53:40.255539 25445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:53:40.255725 25445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:53:40.255928 25445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.195
I20260807 10:53:40.266250 25445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:46781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.195:42781
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.73.195
--webserver_port=44503
--tserver_master_addrs=127.3.73.254:35855
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.195
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:53:40.267791 25445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:53:40.269730 25445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:53:40.284890 25455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:40.285065 25457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:40.285452 25454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:53:40.285765 25445 server_base.cc:1034] running on GCE node
I20260807 10:53:40.286540 25445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:53:40.287698 25445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:53:40.289086 25445 hybrid_clock.cc:648] HybridClock initialized: now 1786100020289026 us; error 51 us; skew 500 ppm
I20260807 10:53:40.289475 25445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:53:40 dist-test-slave-x5fp krb5kdc[24097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100020, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:53:40.293412 25445 init.cc:377] Logged in from keytab as kudu/127.3.73.195@KRBTEST.COM (short username kudu)
I20260807 10:53:40.295181 25445 webserver.cc:460] Webserver started at http://127.3.73.195:44503/ using document root <none> and password file <none>
I20260807 10:53:40.295974 25445 fs_manager.cc:359] Metadata directory not provided
I20260807 10:53:40.296128 25445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:53:40.302476 25445 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260807 10:53:40.305776 25464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:40.307359 25445 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 10:53:40.307564 25445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/wal
uuid: "be67427905c64a048b6def3957fdb418"
format_stamp: "Formatted at 2026-08-07 10:53:37 on dist-test-slave-x5fp"
server_key: "27791662d39c92f7c3f5fab83b1958db"
server_key_iv: "b1b7b729ff3e0a847dc9dc6232aa9ee2"
server_key_version: "encryptionkey@0"
I20260807 10:53:40.308722 25445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:53:40.327557 25445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:53:40.333081 25445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:53:40.333461 25445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:53:40.334851 25445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:53:40.336671 25445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:53:40.336875 25445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 10:53:40.337040 25445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:53:40.337129 25445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:40.363443 25227 heartbeater.cc:503] Master 127.3.73.254:35855 was elected leader, sending a full tablet report...
I20260807 10:53:40.380335 25445 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.195:42781
I20260807 10:53:40.380450 25637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.195:42781 every 8 connection(s)
I20260807 10:53:40.382265 25445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
I20260807 10:53:40.390837 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 25445
Aug 07 10:53:40 dist-test-slave-x5fp krb5kdc[24097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100020, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:53:40.413267 25645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:40.392401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52414 (local address 127.3.73.254:35855)
0807 10:53:40.394477 (+ 2076us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:40.394486 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:40.394520 (+ 34us) server_negotiation.cc:407] Connection header received
0807 10:53:40.394619 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:40.394625 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:40.394741 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:40.395127 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:40.397915 (+ 2788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:40.399253 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:40.401384 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:40.402034 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:40.402190 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:40.405699 (+ 3509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:40.405706 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:40.405709 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:40.405758 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:40.409319 (+ 3561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:40.410169 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:40.410179 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:40.410181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:40.410313 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:40.410755 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:40.410759 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:40.410761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:40.411055 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:40.411518 (+ 463us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:40.412180 (+ 662us) server_negotiation.cc:299] Negotiation successful
0807 10:53:40.412650 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1805,"thread_start_us":1404,"threads_started":1}
I20260807 10:53:40.413609 25640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:40.386099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:54417 (local address 127.3.73.254:35855)
0807 10:53:40.386759 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:40.386765 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:40.387663 (+ 898us) server_negotiation.cc:407] Connection header received
0807 10:53:40.390175 (+ 2512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:40.390181 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:40.390346 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:40.390498 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:40.392666 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:40.395410 (+ 2744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:40.398249 (+ 2839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:40.398779 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:40.398905 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:40.403787 (+ 4882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:40.403793 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:40.403830 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:40.403892 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:40.408388 (+ 4496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:40.409730 (+ 1342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:40.409736 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:40.409739 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:40.409916 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:40.410629 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:40.410633 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:40.410635 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:40.410981 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:40.411165 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:40.412874 (+ 1709us) server_negotiation.cc:299] Negotiation successful
0807 10:53:40.413115 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"spinlock_wait_cycles":6400,"thread_start_us":398,"threads_started":1}
I20260807 10:53:40.414608 25642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:40.386559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.195:54417)
0807 10:53:40.387456 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:40.387513 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 10:53:40.389808 (+ 2295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:53:40.390815 (+ 1007us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:53:40.390826 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:53:40.391592 (+ 766us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:53:40.392444 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:40.392472 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:40.395634 (+ 3162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:40.395639 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:40.397933 (+ 2294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:40.397957 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:40.399153 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:40.399161 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:40.399540 (+ 379us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:40.400364 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:53:40.400391 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:53:40.403493 (+ 3102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:53:40.408619 (+ 5126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:40.408632 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:40.408654 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:40.409561 (+ 907us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:40.410166 (+ 605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:53:40.410171 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:53:40.410175 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:53:40.410463 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:53:40.411157 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:53:40.411166 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:53:40.412065 (+ 899us) client_negotiation.cc:769] Sending connection context
0807 10:53:40.412483 (+ 418us) client_negotiation.cc:240] Negotiation successful
0807 10:53:40.413148 (+ 665us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":351,"threads_started":1}
I20260807 10:53:40.416275 25638 heartbeater.cc:347] Connected to a master server at 127.3.73.254:35855
I20260807 10:53:40.416618 25638 heartbeater.cc:464] Registering TS with master...
I20260807 10:53:40.417550 25638 heartbeater.cc:511] Master 127.3.73.254:35855 requested a full tablet report, sending...
I20260807 10:53:40.419212 24943 ts_manager.cc:194] Registered new tserver with Master: be67427905c64a048b6def3957fdb418 (127.3.73.195:42781)
I20260807 10:53:40.420025 24943 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.195@KRBTEST.COM'} at 127.3.73.195:54417
I20260807 10:53:40.429332 3367 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 10:53:40.437191 25645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:40.430412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52430 (local address 127.3.73.254:35855)
0807 10:53:40.430606 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:40.430611 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:40.431241 (+ 630us) server_negotiation.cc:407] Connection header received
0807 10:53:40.431496 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:40.431500 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:40.431654 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:40.431794 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:53:40.432292 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:40.433081 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:40.434630 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:40.435115 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:40.435230 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:40.435824 (+ 594us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:53:40.436412 (+ 588us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:53:40.436539 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:40.436769 (+ 230us) server_negotiation.cc:299] Negotiation successful
0807 10:53:40.436867 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 10:53:40.438230 24943 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:52430:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 10:53:40.446238 24943 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 10:53:40.483958 25657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:40.461206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38886 (local address 127.3.73.195:42781)
0807 10:53:40.474289 (+ 13083us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:40.474298 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:40.474332 (+ 34us) server_negotiation.cc:407] Connection header received
0807 10:53:40.474447 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:40.474454 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:40.474665 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:40.475812 (+ 1147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:53:40.476112 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:40.477790 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:40.481028 (+ 3238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:40.482568 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:40.482689 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:40.482975 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:40.483405 (+ 430us) server_negotiation.cc:299] Negotiation successful
0807 10:53:40.483597 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12915,"thread_start_us":12730,"threads_started":1}
I20260807 10:53:40.484095 25655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:40.460794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:42781 (local address 127.0.0.1:38886)
0807 10:53:40.473272 (+ 12478us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:40.473300 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0807 10:53:40.473571 (+ 271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:53:40.474873 (+ 1302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:53:40.474878 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:53:40.474913 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:53:40.475571 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:40.475587 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:40.477985 (+ 2398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:40.477991 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:40.480806 (+ 2815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:40.480821 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:40.483006 (+ 2185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:40.483014 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:40.483165 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:40.483199 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 10:53:40.483349 (+ 150us) client_negotiation.cc:240] Negotiation successful
0807 10:53:40.483570 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12277,"mutex_wait_us":10,"spinlock_wait_cycles":15360,"thread_start_us":13394,"threads_started":1}
I20260807 10:53:40.485319 25654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:40.460629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48130 (local address 127.3.73.193:44389)
0807 10:53:40.472466 (+ 11837us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:40.472482 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:40.473371 (+ 889us) server_negotiation.cc:407] Connection header received
0807 10:53:40.473884 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:40.473907 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:40.474179 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:40.474394 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:53:40.476032 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:40.478018 (+ 1986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:40.481734 (+ 3716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:40.483295 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:40.483443 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:40.483892 (+ 449us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:40.484748 (+ 856us) server_negotiation.cc:299] Negotiation successful
0807 10:53:40.484999 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11522,"thread_start_us":11278,"threads_started":1}
I20260807 10:53:40.485325 25653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:40.460463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:44389 (local address 127.0.0.1:48130)
0807 10:53:40.473155 (+ 12692us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:40.473222 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 10:53:40.473572 (+ 350us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:53:40.474859 (+ 1287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:53:40.474871 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:53:40.475000 (+ 129us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:53:40.475787 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:40.475801 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:40.479172 (+ 3371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:40.479177 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:40.481488 (+ 2311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:40.481502 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:40.484100 (+ 2598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:40.484109 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:40.484478 (+ 369us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:40.484528 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 10:53:40.484635 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 10:53:40.484821 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12392,"mutex_wait_us":224,"spinlock_wait_cycles":14720,"thread_start_us":12076,"threads_started":1}
I20260807 10:53:40.488476 25539 tablet_service.cc:1467] Processing CreateTablet for tablet 79b83c69bfb840ec820e6fb60593b671 (DEFAULT_TABLE table=test-table [id=4b9614dad9b34397bb570a59a9c315cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:53:40.489388 25130 tablet_service.cc:1467] Processing CreateTablet for tablet 79b83c69bfb840ec820e6fb60593b671 (DEFAULT_TABLE table=test-table [id=4b9614dad9b34397bb570a59a9c315cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:53:40.490172 25539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79b83c69bfb840ec820e6fb60593b671. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:53:40.490912 25130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79b83c69bfb840ec820e6fb60593b671. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:53:40.491011 25659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:40.475693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:32827 (local address 127.0.0.1:59456)
0807 10:53:40.480101 (+ 4408us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:40.480131 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 10:53:40.480306 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:53:40.480975 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:53:40.480980 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:53:40.481007 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:53:40.481451 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:40.481461 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:40.483864 (+ 2403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:40.483871 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:40.487765 (+ 3894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:40.487786 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:40.489905 (+ 2119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:40.489913 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:40.490254 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:40.490295 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 10:53:40.490419 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 10:53:40.490607 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4252,"thread_start_us":4052,"threads_started":1}
I20260807 10:53:40.491251 25658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:40.473994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59456 (local address 127.3.73.194:32827)
0807 10:53:40.479717 (+ 5723us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:40.479727 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:40.480255 (+ 528us) server_negotiation.cc:407] Connection header received
0807 10:53:40.480493 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:40.480499 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:40.480703 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:40.480995 (+ 292us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:53:40.481911 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:40.483550 (+ 1639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:40.487995 (+ 4445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:40.489621 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:40.489793 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:40.489996 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:40.490474 (+ 478us) server_negotiation.cc:299] Negotiation successful
0807 10:53:40.490661 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5537,"thread_start_us":5392,"threads_started":1}
I20260807 10:53:40.495784 25337 tablet_service.cc:1467] Processing CreateTablet for tablet 79b83c69bfb840ec820e6fb60593b671 (DEFAULT_TABLE table=test-table [id=4b9614dad9b34397bb570a59a9c315cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:53:40.497509 25337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79b83c69bfb840ec820e6fb60593b671. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:53:40.504350 25660 tablet_bootstrap.cc:492] T 79b83c69bfb840ec820e6fb60593b671 P be67427905c64a048b6def3957fdb418: Bootstrap starting.
I20260807 10:53:40.507684 25660 tablet_bootstrap.cc:654] T 79b83c69bfb840ec820e6fb60593b671 P be67427905c64a048b6def3957fdb418: Neither blocks nor log segments found. Creating new log.
I20260807 10:53:40.509666 25661 tablet_bootstrap.cc:492] T 79b83c69bfb840ec820e6fb60593b671 P 3425beb307af44598102bae102debd71: Bootstrap starting.
I20260807 10:53:40.510062 25660 log.cc:824] T 79b83c69bfb840ec820e6fb60593b671 P be67427905c64a048b6def3957fdb418: Log is configured to *not* fsync() on all Append() calls
I20260807 10:53:40.513576 25661 tablet_bootstrap.cc:654] T 79b83c69bfb840ec820e6fb60593b671 P 3425beb307af44598102bae102debd71: Neither blocks nor log segments found. Creating new log.
I20260807 10:53:40.514979 25660 tablet_bootstrap.cc:492] T 79b83c69bfb840ec820e6fb60593b671 P be67427905c64a048b6def3957fdb418: No bootstrap required, opened a new log
I20260807 10:53:40.515210 25661 log.cc:824] T 79b83c69bfb840ec820e6fb60593b671 P 3425beb307af44598102bae102debd71: Log is configured to *not* fsync() on all Append() calls
I20260807 10:53:40.515300 25660 ts_tablet_manager.cc:1397] T 79b83c69bfb840ec820e6fb60593b671 P be67427905c64a048b6def3957fdb418: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260807 10:53:40.515581 25663 tablet_bootstrap.cc:492] T 79b83c69bfb840ec820e6fb60593b671 P 2ad1efccd76440a6b62e006eb4a5f83e: Bootstrap starting.
I20260807 10:53:40.519805 25663 tablet_bootstrap.cc:654] T 79b83c69bfb840ec820e6fb60593b671 P 2ad1efccd76440a6b62e006eb4a5f83e: Neither blocks nor log segments found. Creating new log.
I20260807 10:53:40.521289 25663 log.cc:824] T 79b83c69bfb840ec820e6fb60593b671 P 2ad1efccd76440a6b62e006eb4a5f83e: Log is configured to *not* fsync() on all Append() calls
I20260807 10:53:40.524581 25661 tablet_bootstrap.cc:492] T 79b83c69bfb840ec820e6fb60593b671 P 3425beb307af44598102bae102debd71: No bootstrap required, opened a new log
I20260807 10:53:40.524984 25661 ts_tablet_manager.cc:1397] T 79b83c69bfb840ec820e6fb60593b671 P 3425beb307af44598102bae102debd71: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.004s
I20260807 10:53:40.529279 25660 raft_consensus.cc:348] T 79b83c69bfb840ec820e6fb60593b671 P be67427905c64a048b6def3957fdb418 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3425beb307af44598102bae102debd71" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44389 } } peers { permanent_uuid: "2ad1efccd76440a6b62e006eb4a5f83e" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 32827 } } peers { permanent_uuid: "be67427905c64a048b6def3957fdb418" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 42781 } }
I20260807 10:53:40.529842 25660 raft_consensus.cc:374] T 79b83c69bfb840ec820e6fb60593b671 P be67427905c64a048b6def3957fdb418 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:53:40.529955 25660 raft_consensus.cc:729] T 79b83c69bfb840ec820e6fb60593b671 P be67427905c64a048b6def3957fdb418 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be67427905c64a048b6def3957fdb418, State: Initialized, Role: FOLLOWER
I20260807 10:53:40.530598 25663 tablet_bootstrap.cc:492] T 79b83c69bfb840ec820e6fb60593b671 P 2ad1efccd76440a6b62e006eb4a5f83e: No bootstrap required, opened a new log
I20260807 10:53:40.530707 25660 consensus_queue.cc:260] T 79b83c69bfb840ec820e6fb60593b671 P be67427905c64a048b6def3957fdb418 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3425beb307af44598102bae102debd71" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44389 } } peers { permanent_uuid: "2ad1efccd76440a6b62e006eb4a5f83e" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 32827 } } peers { permanent_uuid: "be67427905c64a048b6def3957fdb418" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 42781 } }
I20260807 10:53:40.530932 25663 ts_tablet_manager.cc:1397] T 79b83c69bfb840ec820e6fb60593b671 P 2ad1efccd76440a6b62e006eb4a5f83e: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.005s
I20260807 10:53:40.532033 25638 heartbeater.cc:503] Master 127.3.73.254:35855 was elected leader, sending a full tablet report...
I20260807 10:53:40.532869 25660 ts_tablet_manager.cc:1428] T 79b83c69bfb840ec820e6fb60593b671 P be67427905c64a048b6def3957fdb418: Time spent starting tablet: real 0.017s user 0.018s sys 0.000s
I20260807 10:53:40.537246 25661 raft_consensus.cc:348] T 79b83c69bfb840ec820e6fb60593b671 P 3425beb307af44598102bae102debd71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3425beb307af44598102bae102debd71" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44389 } } peers { permanent_uuid: "2ad1efccd76440a6b62e006eb4a5f83e" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 32827 } } peers { permanent_uuid: "be67427905c64a048b6def3957fdb418" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 42781 } }
I20260807 10:53:40.537714 25661 raft_consensus.cc:374] T 79b83c69bfb840ec820e6fb60593b671 P 3425beb307af44598102bae102debd71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:53:40.537873 25661 raft_consensus.cc:729] T 79b83c69bfb840ec820e6fb60593b671 P 3425beb307af44598102bae102debd71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3425beb307af44598102bae102debd71, State: Initialized, Role: FOLLOWER
I20260807 10:53:40.538587 25661 consensus_queue.cc:260] T 79b83c69bfb840ec820e6fb60593b671 P 3425beb307af44598102bae102debd71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3425beb307af44598102bae102debd71" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44389 } } peers { permanent_uuid: "2ad1efccd76440a6b62e006eb4a5f83e" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 32827 } } peers { permanent_uuid: "be67427905c64a048b6def3957fdb418" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 42781 } }
I20260807 10:53:40.540858 25661 ts_tablet_manager.cc:1428] T 79b83c69bfb840ec820e6fb60593b671 P 3425beb307af44598102bae102debd71: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
I20260807 10:53:40.543025 25663 raft_consensus.cc:348] T 79b83c69bfb840ec820e6fb60593b671 P 2ad1efccd76440a6b62e006eb4a5f83e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3425beb307af44598102bae102debd71" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44389 } } peers { permanent_uuid: "2ad1efccd76440a6b62e006eb4a5f83e" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 32827 } } peers { permanent_uuid: "be67427905c64a048b6def3957fdb418" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 42781 } }
I20260807 10:53:40.543639 25663 raft_consensus.cc:374] T 79b83c69bfb840ec820e6fb60593b671 P 2ad1efccd76440a6b62e006eb4a5f83e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:53:40.543763 25663 raft_consensus.cc:729] T 79b83c69bfb840ec820e6fb60593b671 P 2ad1efccd76440a6b62e006eb4a5f83e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ad1efccd76440a6b62e006eb4a5f83e, State: Initialized, Role: FOLLOWER
I20260807 10:53:40.544476 25663 consensus_queue.cc:260] T 79b83c69bfb840ec820e6fb60593b671 P 2ad1efccd76440a6b62e006eb4a5f83e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3425beb307af44598102bae102debd71" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44389 } } peers { permanent_uuid: "2ad1efccd76440a6b62e006eb4a5f83e" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 32827 } } peers { permanent_uuid: "be67427905c64a048b6def3957fdb418" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 42781 } }
I20260807 10:53:40.546439 25436 heartbeater.cc:503] Master 127.3.73.254:35855 was elected leader, sending a full tablet report...
I20260807 10:53:40.547233 25663 ts_tablet_manager.cc:1428] T 79b83c69bfb840ec820e6fb60593b671 P 2ad1efccd76440a6b62e006eb4a5f83e: Time spent starting tablet: real 0.016s user 0.008s sys 0.008s
W20260807 10:53:40.569150 25228 tablet.cc:2369] T 79b83c69bfb840ec820e6fb60593b671 P 3425beb307af44598102bae102debd71: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 10:53:40.602926 25439 tablet.cc:2369] T 79b83c69bfb840ec820e6fb60593b671 P 2ad1efccd76440a6b62e006eb4a5f83e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 10:53:40.634527 25639 tablet.cc:2369] T 79b83c69bfb840ec820e6fb60593b671 P be67427905c64a048b6def3957fdb418: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:53:40.697628 25667 raft_consensus.cc:482] T 79b83c69bfb840ec820e6fb60593b671 P 3425beb307af44598102bae102debd71 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 10:53:40.698033 25667 raft_consensus.cc:504] T 79b83c69bfb840ec820e6fb60593b671 P 3425beb307af44598102bae102debd71 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3425beb307af44598102bae102debd71" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44389 } } peers { permanent_uuid: "2ad1efccd76440a6b62e006eb4a5f83e" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 32827 } } peers { permanent_uuid: "be67427905c64a048b6def3957fdb418" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 42781 } }
I20260807 10:53:40.700269 25667 leader_election.cc:288] T 79b83c69bfb840ec820e6fb60593b671 P 3425beb307af44598102bae102debd71 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ad1efccd76440a6b62e006eb4a5f83e (127.3.73.194:32827), be67427905c64a048b6def3957fdb418 (127.3.73.195:42781)
I20260807 10:53:40.710585 25658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:40.700868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:44093 (local address 127.3.73.194:32827)
0807 10:53:40.701109 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:40.701117 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:40.701951 (+ 834us) server_negotiation.cc:407] Connection header received
0807 10:53:40.702199 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:40.702205 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:40.702285 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:40.702394 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:53:40.703397 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:40.704591 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:40.707951 (+ 3360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:40.709379 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:40.709578 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:40.709683 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:40.710121 (+ 438us) server_negotiation.cc:299] Negotiation successful
0807 10:53:40.710301 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 10:53:40.710585 25657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:40.700934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:56699 (local address 127.3.73.195:42781)
0807 10:53:40.701120 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:40.701127 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:40.701951 (+ 824us) server_negotiation.cc:407] Connection header received
0807 10:53:40.702178 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:40.702185 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:40.702277 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:40.702394 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:53:40.703378 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:40.704591 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:40.707951 (+ 3360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:40.709529 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:40.709705 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:40.709784 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:40.710126 (+ 342us) server_negotiation.cc:299] Negotiation successful
0807 10:53:40.710272 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 10:53:40.710747 25671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:40.700650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:42781 (local address 127.3.73.193:56699)
0807 10:53:40.701792 (+ 1142us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:40.701813 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 10:53:40.702033 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:53:40.702438 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:53:40.702443 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:53:40.702494 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:53:40.703144 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:40.703167 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:40.704794 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:40.704800 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:40.707695 (+ 2895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:40.707713 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:40.709742 (+ 2029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:40.709748 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:40.709946 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:40.709990 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 10:53:40.710073 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 10:53:40.710186 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":966,"mutex_wait_us":178,"spinlock_wait_cycles":55424,"thread_start_us":570,"threads_started":1}
I20260807 10:53:40.711248 25676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:40.700704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:32827 (local address 127.3.73.193:44093)
0807 10:53:40.701792 (+ 1088us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:40.701813 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 10:53:40.701984 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:53:40.702438 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:53:40.702443 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:53:40.702510 (+ 67us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:53:40.703186 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:40.703195 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:40.704794 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:40.704800 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:40.707695 (+ 2895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:53:40.707713 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:53:40.709650 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:53:40.709660 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:53:40.709862 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:40.709908 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 10:53:40.710022 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 10:53:40.710186 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":971,"mutex_wait_us":355,"spinlock_wait_cycles":54656,"thread_start_us":539,"threads_started":1}
I20260807 10:53:40.712265 25575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79b83c69bfb840ec820e6fb60593b671" candidate_uuid: "3425beb307af44598102bae102debd71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be67427905c64a048b6def3957fdb418" is_pre_election: true
I20260807 10:53:40.712359 25370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79b83c69bfb840ec820e6fb60593b671" candidate_uuid: "3425beb307af44598102bae102debd71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ad1efccd76440a6b62e006eb4a5f83e" is_pre_election: true
I20260807 10:53:40.713011 25370 raft_consensus.cc:2454] T 79b83c69bfb840ec820e6fb60593b671 P 2ad1efccd76440a6b62e006eb4a5f83e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3425beb307af44598102bae102debd71 in term 0.
I20260807 10:53:40.713011 25575 raft_consensus.cc:2454] T 79b83c69bfb840ec820e6fb60593b671 P be67427905c64a048b6def3957fdb418 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3425beb307af44598102bae102debd71 in term 0.
I20260807 10:53:40.713864 25058 leader_election.cc:302] T 79b83c69bfb840ec820e6fb60593b671 P 3425beb307af44598102bae102debd71 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ad1efccd76440a6b62e006eb4a5f83e, 3425beb307af44598102bae102debd71; no voters:
I20260807 10:53:40.714330 25667 raft_consensus.cc:2781] T 79b83c69bfb840ec820e6fb60593b671 P 3425beb307af44598102bae102debd71 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 10:53:40.714502 25667 raft_consensus.cc:482] T 79b83c69bfb840ec820e6fb60593b671 P 3425beb307af44598102bae102debd71 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 10:53:40.714613 25667 raft_consensus.cc:3037] T 79b83c69bfb840ec820e6fb60593b671 P 3425beb307af44598102bae102debd71 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:53:40.717047 25667 raft_consensus.cc:504] T 79b83c69bfb840ec820e6fb60593b671 P 3425beb307af44598102bae102debd71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3425beb307af44598102bae102debd71" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44389 } } peers { permanent_uuid: "2ad1efccd76440a6b62e006eb4a5f83e" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 32827 } } peers { permanent_uuid: "be67427905c64a048b6def3957fdb418" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 42781 } }
I20260807 10:53:40.717778 25667 leader_election.cc:288] T 79b83c69bfb840ec820e6fb60593b671 P 3425beb307af44598102bae102debd71 [CANDIDATE]: Term 1 election: Requested vote from peers 2ad1efccd76440a6b62e006eb4a5f83e (127.3.73.194:32827), be67427905c64a048b6def3957fdb418 (127.3.73.195:42781)
I20260807 10:53:40.718389 25575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79b83c69bfb840ec820e6fb60593b671" candidate_uuid: "3425beb307af44598102bae102debd71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be67427905c64a048b6def3957fdb418"
I20260807 10:53:40.718400 25370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79b83c69bfb840ec820e6fb60593b671" candidate_uuid: "3425beb307af44598102bae102debd71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ad1efccd76440a6b62e006eb4a5f83e"
I20260807 10:53:40.718616 25575 raft_consensus.cc:3037] T 79b83c69bfb840ec820e6fb60593b671 P be67427905c64a048b6def3957fdb418 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:53:40.718644 25370 raft_consensus.cc:3037] T 79b83c69bfb840ec820e6fb60593b671 P 2ad1efccd76440a6b62e006eb4a5f83e [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:53:40.721625 25575 raft_consensus.cc:2454] T 79b83c69bfb840ec820e6fb60593b671 P be67427905c64a048b6def3957fdb418 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3425beb307af44598102bae102debd71 in term 1.
I20260807 10:53:40.721815 25370 raft_consensus.cc:2454] T 79b83c69bfb840ec820e6fb60593b671 P 2ad1efccd76440a6b62e006eb4a5f83e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3425beb307af44598102bae102debd71 in term 1.
I20260807 10:53:40.722153 25056 leader_election.cc:302] T 79b83c69bfb840ec820e6fb60593b671 P 3425beb307af44598102bae102debd71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3425beb307af44598102bae102debd71, be67427905c64a048b6def3957fdb418; no voters:
I20260807 10:53:40.722563 25667 raft_consensus.cc:2781] T 79b83c69bfb840ec820e6fb60593b671 P 3425beb307af44598102bae102debd71 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:53:40.723150 25667 raft_consensus.cc:686] T 79b83c69bfb840ec820e6fb60593b671 P 3425beb307af44598102bae102debd71 [term 1 LEADER]: Becoming Leader. State: Replica: 3425beb307af44598102bae102debd71, State: Running, Role: LEADER
I20260807 10:53:40.723762 25667 consensus_queue.cc:237] T 79b83c69bfb840ec820e6fb60593b671 P 3425beb307af44598102bae102debd71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3425beb307af44598102bae102debd71" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44389 } } peers { permanent_uuid: "2ad1efccd76440a6b62e006eb4a5f83e" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 32827 } } peers { permanent_uuid: "be67427905c64a048b6def3957fdb418" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 42781 } }
I20260807 10:53:40.728993 24943 catalog_manager.cc:5495] T 79b83c69bfb840ec820e6fb60593b671 P 3425beb307af44598102bae102debd71 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3425beb307af44598102bae102debd71 (127.3.73.193). New cstate: current_term: 1 leader_uuid: "3425beb307af44598102bae102debd71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3425beb307af44598102bae102debd71" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 44389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ad1efccd76440a6b62e006eb4a5f83e" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 32827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "be67427905c64a048b6def3957fdb418" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 42781 } health_report { overall_health: UNKNOWN } } }
I20260807 10:53:40.790902 25654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:40.785232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48142 (local address 127.3.73.193:44389)
0807 10:53:40.785515 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:40.785521 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:40.785540 (+ 19us) server_negotiation.cc:407] Connection header received
0807 10:53:40.785634 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:40.785638 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:40.785730 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:40.785867 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:53:40.786323 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:40.787114 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:40.788594 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:40.789118 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:40.789283 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:40.789741 (+ 458us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:53:40.790285 (+ 544us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:53:40.790403 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:40.790527 (+ 124us) server_negotiation.cc:299] Negotiation successful
0807 10:53:40.790597 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260807 10:53:40.799482 25575 raft_consensus.cc:1261] T 79b83c69bfb840ec820e6fb60593b671 P be67427905c64a048b6def3957fdb418 [term 1 FOLLOWER]: Refusing update from remote peer 3425beb307af44598102bae102debd71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:53:40.799482 25370 raft_consensus.cc:1261] T 79b83c69bfb840ec820e6fb60593b671 P 2ad1efccd76440a6b62e006eb4a5f83e [term 1 FOLLOWER]: Refusing update from remote peer 3425beb307af44598102bae102debd71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:53:40.800515 25677 consensus_queue.cc:1035] T 79b83c69bfb840ec820e6fb60593b671 P 3425beb307af44598102bae102debd71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ad1efccd76440a6b62e006eb4a5f83e" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 32827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:53:40.800995 25667 consensus_queue.cc:1035] T 79b83c69bfb840ec820e6fb60593b671 P 3425beb307af44598102bae102debd71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be67427905c64a048b6def3957fdb418" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 42781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:53:40.816576 25683 mvcc.cc:205] Tried to move back new op lower bound from 7315865685184655360 to 7315865684889165824. Current Snapshot: MvccSnapshot[applied={T|T < 7315865685184655360}]
I20260807 10:53:40.821020 25685 mvcc.cc:205] Tried to move back new op lower bound from 7315865685184655360 to 7315865684889165824. Current Snapshot: MvccSnapshot[applied={T|T < 7315865685184655360}]
I20260807 10:53:40.821887 25681 mvcc.cc:205] Tried to move back new op lower bound from 7315865685184655360 to 7315865684889165824. Current Snapshot: MvccSnapshot[applied={T|T < 7315865685184655360 or (T in {7315865685184655360})}]
I20260807 10:53:40.830894 24943 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:52430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:53:40.831211 24943 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:52430:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:53:40.835146 24943 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05: Sending DeleteTablet for 3 replicas of tablet 79b83c69bfb840ec820e6fb60593b671
I20260807 10:53:40.836681 25337 tablet_service.cc:1514] Processing DeleteTablet for tablet 79b83c69bfb840ec820e6fb60593b671 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:53:40 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:59456
I20260807 10:53:40.836689 25539 tablet_service.cc:1514] Processing DeleteTablet for tablet 79b83c69bfb840ec820e6fb60593b671 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:53:40 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:38886
I20260807 10:53:40.837519 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 24999
I20260807 10:53:40.837549 25130 tablet_service.cc:1514] Processing DeleteTablet for tablet 79b83c69bfb840ec820e6fb60593b671 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:53:40 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:48130
I20260807 10:53:40.838588 25696 tablet_replica.cc:330] T 79b83c69bfb840ec820e6fb60593b671 P be67427905c64a048b6def3957fdb418: stopping tablet replica
I20260807 10:53:40.839063 25696 raft_consensus.cc:2229] T 79b83c69bfb840ec820e6fb60593b671 P be67427905c64a048b6def3957fdb418 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:53:40.839653 25696 raft_consensus.cc:2258] T 79b83c69bfb840ec820e6fb60593b671 P be67427905c64a048b6def3957fdb418 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:53:40.840756 25698 tablet_replica.cc:330] T 79b83c69bfb840ec820e6fb60593b671 P 3425beb307af44598102bae102debd71: stopping tablet replica
I20260807 10:53:40.841219 25698 raft_consensus.cc:2229] T 79b83c69bfb840ec820e6fb60593b671 P 3425beb307af44598102bae102debd71 [term 1 LEADER]: Raft consensus shutting down.
I20260807 10:53:40.841926 25698 raft_consensus.cc:2258] T 79b83c69bfb840ec820e6fb60593b671 P 3425beb307af44598102bae102debd71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:53:40.842718 25696 ts_tablet_manager.cc:1905] T 79b83c69bfb840ec820e6fb60593b671 P be67427905c64a048b6def3957fdb418: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:53:40.842716 25695 tablet_replica.cc:330] T 79b83c69bfb840ec820e6fb60593b671 P 2ad1efccd76440a6b62e006eb4a5f83e: stopping tablet replica
I20260807 10:53:40.843179 25695 raft_consensus.cc:2229] T 79b83c69bfb840ec820e6fb60593b671 P 2ad1efccd76440a6b62e006eb4a5f83e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:53:40.843729 25695 raft_consensus.cc:2258] T 79b83c69bfb840ec820e6fb60593b671 P 2ad1efccd76440a6b62e006eb4a5f83e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:53:40.847802 25698 ts_tablet_manager.cc:1905] T 79b83c69bfb840ec820e6fb60593b671 P 3425beb307af44598102bae102debd71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:53:40.851845 25695 ts_tablet_manager.cc:1905] T 79b83c69bfb840ec820e6fb60593b671 P 2ad1efccd76440a6b62e006eb4a5f83e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:53:40.859524 25696 ts_tablet_manager.cc:1918] T 79b83c69bfb840ec820e6fb60593b671 P be67427905c64a048b6def3957fdb418: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:53:40.859818 25696 log.cc:1196] T 79b83c69bfb840ec820e6fb60593b671 P be67427905c64a048b6def3957fdb418: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/wal/wals/79b83c69bfb840ec820e6fb60593b671
I20260807 10:53:40.860416 25696 ts_tablet_manager.cc:1939] T 79b83c69bfb840ec820e6fb60593b671 P be67427905c64a048b6def3957fdb418: Deleting consensus metadata
I20260807 10:53:40.861043 25654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:40.839266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48144 (local address 127.3.73.193:44389)
0807 10:53:40.839448 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:40.839454 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:40.839473 (+ 19us) server_negotiation.cc:407] Connection header received
0807 10:53:40.839532 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:40.839537 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:40.839636 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:40.839744 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:40.842087 (+ 2343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:40.843271 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:40.845602 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:40.846303 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:40.846453 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:40.850588 (+ 4135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:40.850605 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:40.850622 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:40.850671 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:40.855112 (+ 4441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:40.856088 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:40.856098 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:40.856102 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:40.856280 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:40.856685 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:40.856692 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:40.856696 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:40.857243 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:40.857591 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:40.860185 (+ 2594us) server_negotiation.cc:299] Negotiation successful
0807 10:53:40.860586 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 10:53:40.862039 25222 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:53:40.862507 25695 ts_tablet_manager.cc:1918] T 79b83c69bfb840ec820e6fb60593b671 P 2ad1efccd76440a6b62e006eb4a5f83e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:53:40.862684 25695 log.cc:1196] T 79b83c69bfb840ec820e6fb60593b671 P 2ad1efccd76440a6b62e006eb4a5f83e: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/wal/wals/79b83c69bfb840ec820e6fb60593b671
I20260807 10:53:40.862618 24914 catalog_manager.cc:4841] TS be67427905c64a048b6def3957fdb418 (127.3.73.195:42781): tablet 79b83c69bfb840ec820e6fb60593b671 (table test-table [id=4b9614dad9b34397bb570a59a9c315cc]) successfully deleted
I20260807 10:53:40.863300 25695 ts_tablet_manager.cc:1939] T 79b83c69bfb840ec820e6fb60593b671 P 2ad1efccd76440a6b62e006eb4a5f83e: Deleting consensus metadata
I20260807 10:53:40.865314 24918 catalog_manager.cc:4841] TS 2ad1efccd76440a6b62e006eb4a5f83e (127.3.73.194:32827): tablet 79b83c69bfb840ec820e6fb60593b671 (table test-table [id=4b9614dad9b34397bb570a59a9c315cc]) successfully deleted
I20260807 10:53:40.971475 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 24999
I20260807 10:53:40.996683 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 25235
W20260807 10:53:40.996654 24918 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.73.193:44389 (error 108)
I20260807 10:53:41.001246 25704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:40.998189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:44389 (local address 127.0.0.1:48150)
0807 10:53:41.000922 (+ 2733us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:41.001061 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.193:44389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2587,"thread_start_us":2471,"threads_started":1}
W20260807 10:53:41.001507 24918 catalog_manager.cc:4584] TS 3425beb307af44598102bae102debd71 (127.3.73.193:44389): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 79b83c69bfb840ec820e6fb60593b671: Network error: Client connection negotiation failed: client connection to 127.3.73.193:44389: connect: Connection refused (error 111)
I20260807 10:53:41.019385 25658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:40.997686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59460 (local address 127.3.73.194:32827)
0807 10:53:40.997903 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:40.997910 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:40.997959 (+ 49us) server_negotiation.cc:407] Connection header received
0807 10:53:40.998018 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:40.998022 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:40.998088 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:40.998162 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:41.000555 (+ 2393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:41.001533 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:41.003254 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:41.003675 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:41.003790 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:41.011744 (+ 7954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:41.011766 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:41.011785 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:41.011837 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:41.015626 (+ 3789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:41.016694 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:41.016706 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:41.016710 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:41.016985 (+ 275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:41.017515 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:41.017520 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:41.017523 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:41.018105 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:41.018359 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:41.018806 (+ 447us) server_negotiation.cc:299] Negotiation successful
0807 10:53:41.019033 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 10:53:41.020280 25427 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:53:41.055312 25704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:41.054694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:44389 (local address 127.0.0.1:48158)
0807 10:53:41.054969 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:41.055107 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.193:44389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260807 10:53:41.056478 25704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:41.056060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:44389 (local address 127.0.0.1:48174)
0807 10:53:41.056257 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:41.056329 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.193:44389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260807 10:53:41.115914 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 25235
I20260807 10:53:41.119860 25704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:41.119367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:44389 (local address 127.0.0.1:48178)
0807 10:53:41.119587 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:41.119701 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.193:44389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260807 10:53:41.121034 25704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:41.120623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:44389 (local address 127.0.0.1:48188)
0807 10:53:41.120797 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:41.120888 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.193:44389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 10:53:41.146458 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 25445
I20260807 10:53:41.162365 25657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:41.147354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38888 (local address 127.3.73.195:42781)
0807 10:53:41.147537 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:41.147542 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:41.147563 (+ 21us) server_negotiation.cc:407] Connection header received
0807 10:53:41.147669 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:41.147673 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:41.147741 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:41.147830 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:41.150147 (+ 2317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:41.150903 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:41.152446 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:41.153049 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:41.153167 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:41.155988 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:41.156004 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:41.156021 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:41.156061 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:41.159369 (+ 3308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:41.160208 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:41.160217 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:41.160219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:41.160378 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:41.160824 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:41.160828 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:41.160830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:41.161244 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:41.161482 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:41.161831 (+ 349us) server_negotiation.cc:299] Negotiation successful
0807 10:53:41.162037 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 10:53:41.163122 25629 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:53:41.201902 25704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:41.201309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:44389 (local address 127.0.0.1:48198)
0807 10:53:41.201578 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:41.201679 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.193:44389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260807 10:53:41.203329 25704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:41.202713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:44389 (local address 127.0.0.1:48204)
0807 10:53:41.203086 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:41.203164 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.193:44389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274}
I20260807 10:53:41.259084 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 25445
I20260807 10:53:41.282747 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 24884
I20260807 10:53:41.284184 24989 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:53:41.406343 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 24884
I20260807 10:53:41.406417 25704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:41.405684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:44389 (local address 127.0.0.1:48210)
0807 10:53:41.406123 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:41.406255 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.193:44389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231}
I20260807 10:53:41.440022 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:35855
--webserver_interface=127.3.73.254
--webserver_port=39523
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:46781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:35855
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:53:41.825464 25712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:53:41.825989 25712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:53:41.826169 25712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:53:41.835639 25712 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:53:41.835817 25712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:53:41.835904 25712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:53:41.836019 25712 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:53:41.836109 25712 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:53:41.846913 25712 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:46781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:35855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:35855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=39523
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:53:41.848454 25712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:53:41.849949 25712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:53:41.864167 25720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:41.864253 25722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:41.864738 25719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:53:41.864948 25712 server_base.cc:1034] running on GCE node
I20260807 10:53:41.866220 25712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:53:41.867790 25712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:53:41.869041 25712 hybrid_clock.cc:648] HybridClock initialized: now 1786100021868975 us; error 63 us; skew 500 ppm
I20260807 10:53:41.869412 25712 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:53:41 dist-test-slave-x5fp krb5kdc[24097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100021, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:53:41.873133 25712 init.cc:377] Logged in from keytab as kudu/127.3.73.254@KRBTEST.COM (short username kudu)
I20260807 10:53:41.874918 25712 webserver.cc:460] Webserver started at http://127.3.73.254:39523/ using document root <none> and password file <none>
I20260807 10:53:41.875689 25712 fs_manager.cc:359] Metadata directory not provided
I20260807 10:53:41.875850 25712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:53:41.882288 25712 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 10:53:41.888432 25734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:41.890414 25712 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 10:53:41.890656 25712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "e9c8d1c8814f4b03b923ff6cdc695c05"
format_stamp: "Formatted at 2026-08-07 10:53:35 on dist-test-slave-x5fp"
server_key: "4a762993e8207c6ceee90645f7869f5c"
server_key_iv: "8601664132f7a6639e84a2aaa65819e7"
server_key_version: "encryptionkey@0"
I20260807 10:53:41.891785 25712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:53:41.909379 25712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:53:41.914561 25712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:53:41.914965 25712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:53:41.936959 25712 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:35855
I20260807 10:53:41.936978 25808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:35855 every 8 connection(s)
I20260807 10:53:41.938676 25712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:53:41.946060 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 25712
I20260807 10:53:41.947192 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.193:44389
--local_ip_for_outbound_sockets=127.3.73.193
--tserver_master_addrs=127.3.73.254:35855
--webserver_port=45273
--webserver_interface=127.3.73.193
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:46781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 10:53:41.950947 25810 sys_catalog.cc:263] Verifying existing consensus state
I20260807 10:53:41.957230 25810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05: Bootstrap starting.
I20260807 10:53:41.992359 25810 log.cc:824] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05: Log is configured to *not* fsync() on all Append() calls
I20260807 10:53:42.019876 25810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 10:53:42.020843 25810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05: Bootstrap complete.
I20260807 10:53:42.037259 25810 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9c8d1c8814f4b03b923ff6cdc695c05" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 35855 } }
I20260807 10:53:42.038667 25810 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9c8d1c8814f4b03b923ff6cdc695c05, State: Initialized, Role: FOLLOWER
I20260807 10:53:42.039466 25810 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9c8d1c8814f4b03b923ff6cdc695c05" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 35855 } }
I20260807 10:53:42.039785 25810 raft_consensus.cc:388] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:53:42.039939 25810 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:53:42.040156 25810 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [term 2 FOLLOWER]: Advancing to term 3
I20260807 10:53:42.046334 25810 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9c8d1c8814f4b03b923ff6cdc695c05" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 35855 } }
I20260807 10:53:42.047242 25810 leader_election.cc:302] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9c8d1c8814f4b03b923ff6cdc695c05; no voters:
I20260807 10:53:42.048174 25810 leader_election.cc:288] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 10:53:42.048333 25819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 10:53:42.051231 25819 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [term 3 LEADER]: Becoming Leader. State: Replica: e9c8d1c8814f4b03b923ff6cdc695c05, State: Running, Role: LEADER
I20260807 10:53:42.052011 25819 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9c8d1c8814f4b03b923ff6cdc695c05" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 35855 } }
I20260807 10:53:42.052544 25810 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:53:42.063012 25823 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "e9c8d1c8814f4b03b923ff6cdc695c05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9c8d1c8814f4b03b923ff6cdc695c05" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 35855 } } }
I20260807 10:53:42.063526 25823 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [sys.catalog]: This master's current role is: LEADER
I20260807 10:53:42.077663 25825 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9c8d1c8814f4b03b923ff6cdc695c05. Latest consensus state: current_term: 3 leader_uuid: "e9c8d1c8814f4b03b923ff6cdc695c05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9c8d1c8814f4b03b923ff6cdc695c05" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 35855 } } }
I20260807 10:53:42.078322 25825 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9c8d1c8814f4b03b923ff6cdc695c05 [sys.catalog]: This master's current role is: LEADER
I20260807 10:53:42.079452 25832 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:53:42.091782 25832 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:53:42.094205 25832 catalog_manager.cc:1259] Loaded cluster ID: e4f39d529d4e4402bd5703345d6a4a30
I20260807 10:53:42.094504 25832 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 10:53:42.097560 25832 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f0515b579ad google::LogMessage::Fail() at ??:0
@ 0x7f0515b5a88a google::LogMessage::SendToLog() at ??:0
@ 0x7f0515b574de google::LogMessage::Flush() at ??:0
@ 0x7f0515b58a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f052b58e512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f052b58b976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f052b613610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f052b613439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f05187a3345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f05187a691c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f05187a6799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f0518781eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f051ba5b6db start_thread at ??:0
@ 0x7f05149f161f clone at ??:0
W20260807 10:53:42.414057 25813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:53:42.414386 25813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:53:42.414489 25813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:53:42.422545 25813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:53:42.422756 25813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:53:42.422986 25813 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.193
I20260807 10:53:42.434057 25813 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:46781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.193:44389
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.73.193
--webserver_port=45273
--tserver_master_addrs=127.3.73.254:35855
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.193
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:53:42.435570 25813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:53:42.437173 25813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:53:42.455320 25875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:42.455706 25879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:42.455345 25872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:53:42.456810 25813 server_base.cc:1034] running on GCE node
I20260807 10:53:42.457581 25813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:53:42.458814 25813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:53:42.460167 25813 hybrid_clock.cc:648] HybridClock initialized: now 1786100022460078 us; error 66 us; skew 500 ppm
I20260807 10:53:42.460575 25813 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:53:42 dist-test-slave-x5fp krb5kdc[24097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100022, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:53:42.464771 25813 init.cc:377] Logged in from keytab as kudu/127.3.73.193@KRBTEST.COM (short username kudu)
I20260807 10:53:42.466938 25813 webserver.cc:460] Webserver started at http://127.3.73.193:45273/ using document root <none> and password file <none>
I20260807 10:53:42.467846 25813 fs_manager.cc:359] Metadata directory not provided
I20260807 10:53:42.468078 25813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:53:42.475001 25813 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.003s
I20260807 10:53:42.479198 25888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:42.481243 25813 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 10:53:42.481511 25813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/wal
uuid: "3425beb307af44598102bae102debd71"
format_stamp: "Formatted at 2026-08-07 10:53:35 on dist-test-slave-x5fp"
server_key: "7d2261ea05a9dba3968af1cd990cb68d"
server_key_iv: "57de6d69b5f35dacf8f12537c6805b32"
server_key_version: "encryptionkey@0"
I20260807 10:53:42.482759 25813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:53:42.504276 25813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:53:42.509684 25813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:53:42.510015 25813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:53:42.511138 25813 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:53:42.513479 25904 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260807 10:53:42.519770 25904 ts_tablet_manager.cc:1852] T 79b83c69bfb840ec820e6fb60593b671 P 3425beb307af44598102bae102debd71: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260807 10:53:42.521122 25904 ts_tablet_manager.cc:1905] T 79b83c69bfb840ec820e6fb60593b671 P 3425beb307af44598102bae102debd71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:53:42.530051 25904 ts_tablet_manager.cc:1918] T 79b83c69bfb840ec820e6fb60593b671 P 3425beb307af44598102bae102debd71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:53:42.530332 25904 log.cc:1196] T 79b83c69bfb840ec820e6fb60593b671 P 3425beb307af44598102bae102debd71: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/wal/wals/79b83c69bfb840ec820e6fb60593b671
I20260807 10:53:42.530864 25904 ts_tablet_manager.cc:1939] T 79b83c69bfb840ec820e6fb60593b671 P 3425beb307af44598102bae102debd71: Deleting consensus metadata
I20260807 10:53:42.531347 25813 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260807 10:53:42.531450 25813 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.018s user 0.001s sys 0.000s
I20260807 10:53:42.531641 25813 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:53:42.531709 25813 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:42.571473 25813 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.193:44389
I20260807 10:53:42.571574 26067 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.193:44389 every 8 connection(s)
I20260807 10:53:42.573099 25813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
I20260807 10:53:42.575552 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 25813
I20260807 10:53:42.576236 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.194:32827
--local_ip_for_outbound_sockets=127.3.73.194
--tserver_master_addrs=127.3.73.254:35855
--webserver_port=42997
--webserver_interface=127.3.73.194
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:46781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 10:53:42.579083 26070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:42.576650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.193:57693)
0807 10:53:42.577710 (+ 1060us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:42.578043 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":729,"thread_start_us":440,"threads_started":1}
W20260807 10:53:42.579914 26068 heartbeater.cc:644] Failed to heartbeat to 127.3.73.254:35855 (0 consecutive failures): Network error: Failed to ping master at 127.3.73.254:35855: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
I20260807 10:53:42.581024 26070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:42.580585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.193:55861)
0807 10:53:42.580815 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:42.580901 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260807 10:53:42.581998 26070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:42.581625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.193:36407)
0807 10:53:42.581788 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:42.581873 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260807 10:53:42.582216 26068 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 10:53:42.950824 26071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:53:42.951156 26071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:53:42.951342 26071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:53:42.959609 26071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:53:42.959830 26071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:53:42.960083 26071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.194
I20260807 10:53:42.972334 26071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:46781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.194:32827
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.73.194
--webserver_port=42997
--tserver_master_addrs=127.3.73.254:35855
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.194
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:53:42.973964 26071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:53:42.975544 26071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:53:42.991566 26079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:42.991591 26080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:42.991516 26082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:53:42.997742 26071 server_base.cc:1034] running on GCE node
I20260807 10:53:42.998633 26071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:53:42.999848 26071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:53:43.001107 26071 hybrid_clock.cc:648] HybridClock initialized: now 1786100023001060 us; error 35 us; skew 500 ppm
I20260807 10:53:43.001529 26071 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:53:42 dist-test-slave-x5fp krb5kdc[24097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100022, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:53:43.005144 26071 init.cc:377] Logged in from keytab as kudu/127.3.73.194@KRBTEST.COM (short username kudu)
I20260807 10:53:43.006908 26071 webserver.cc:460] Webserver started at http://127.3.73.194:42997/ using document root <none> and password file <none>
I20260807 10:53:43.007752 26071 fs_manager.cc:359] Metadata directory not provided
I20260807 10:53:43.007920 26071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:53:43.014025 26071 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 10:53:43.017454 26093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:43.019070 26071 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 10:53:43.019333 26071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/wal
uuid: "2ad1efccd76440a6b62e006eb4a5f83e"
format_stamp: "Formatted at 2026-08-07 10:53:36 on dist-test-slave-x5fp"
server_key: "a578eea0239e45fb35269d810a4448bc"
server_key_iv: "941c49b2167c89b10e5f364a357476b9"
server_key_version: "encryptionkey@0"
I20260807 10:53:43.020546 26071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:53:43.035122 26071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:53:43.040304 26071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:53:43.040661 26071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:53:43.041942 26071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:53:43.043563 26071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:53:43.043715 26071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:43.043865 26071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:53:43.043968 26071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:43.084195 26071 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.194:32827
I20260807 10:53:43.084275 26271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.194:32827 every 8 connection(s)
I20260807 10:53:43.086056 26071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
I20260807 10:53:43.092057 26276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:43.089829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.194:52033)
0807 10:53:43.090725 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:43.091004 (+ 279us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":371,"threads_started":1}
W20260807 10:53:43.093245 26272 heartbeater.cc:644] Failed to heartbeat to 127.3.73.254:35855 (0 consecutive failures): Network error: Failed to ping master at 127.3.73.254:35855: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
I20260807 10:53:43.093398 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 26071
I20260807 10:53:43.094089 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.195:42781
--local_ip_for_outbound_sockets=127.3.73.195
--tserver_master_addrs=127.3.73.254:35855
--webserver_port=44503
--webserver_interface=127.3.73.195
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:46781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 10:53:43.094439 26276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:43.093957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.194:39633)
0807 10:53:43.094140 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:43.094270 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 10:53:43.095582 26276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:43.095122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.194:43019)
0807 10:53:43.095334 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:43.095440 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
W20260807 10:53:43.095856 26272 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 10:53:43.469226 26277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:53:43.469569 26277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:53:43.469672 26277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:53:43.477572 26277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:53:43.477705 26277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:53:43.477831 26277 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.195
I20260807 10:53:43.488530 26277 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:46781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.195:42781
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.73.195
--webserver_port=44503
--tserver_master_addrs=127.3.73.254:35855
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.195
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:53:43.490008 26277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:53:43.491359 26277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:53:43.506258 26282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:43.506227 26283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:53:43.506484 26285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:53:43.507615 26277 server_base.cc:1034] running on GCE node
I20260807 10:53:43.508486 26277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:53:43.509707 26277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:53:43.510962 26277 hybrid_clock.cc:648] HybridClock initialized: now 1786100023510899 us; error 51 us; skew 500 ppm
I20260807 10:53:43.511346 26277 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:53:43 dist-test-slave-x5fp krb5kdc[24097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100023, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:53:43.515045 26277 init.cc:377] Logged in from keytab as kudu/127.3.73.195@KRBTEST.COM (short username kudu)
I20260807 10:53:43.516845 26277 webserver.cc:460] Webserver started at http://127.3.73.195:44503/ using document root <none> and password file <none>
I20260807 10:53:43.517639 26277 fs_manager.cc:359] Metadata directory not provided
I20260807 10:53:43.517804 26277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:53:43.523985 26277 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 10:53:43.527407 26298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:43.529112 26277 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 10:53:43.529338 26277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/wal
uuid: "be67427905c64a048b6def3957fdb418"
format_stamp: "Formatted at 2026-08-07 10:53:37 on dist-test-slave-x5fp"
server_key: "27791662d39c92f7c3f5fab83b1958db"
server_key_iv: "b1b7b729ff3e0a847dc9dc6232aa9ee2"
server_key_version: "encryptionkey@0"
I20260807 10:53:43.530431 26277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:53:43.549458 26277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:53:43.554641 26277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:53:43.554972 26277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:53:43.556133 26277 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:53:43.557729 26277 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:53:43.557873 26277 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:43.558027 26277 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:53:43.558132 26277 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:53:43.584368 26399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:43.583030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.193:46465)
0807 10:53:43.583997 (+ 967us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:43.584114 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":836,"thread_start_us":719,"threads_started":1}
I20260807 10:53:43.598340 26277 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.195:42781
I20260807 10:53:43.598421 26459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.195:42781 every 8 connection(s)
I20260807 10:53:43.600127 26277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
I20260807 10:53:43.606014 26465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:43.603642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.195:44399)
0807 10:53:43.604562 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:43.605048 (+ 486us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":365,"threads_started":1}
W20260807 10:53:43.606956 26461 heartbeater.cc:644] Failed to heartbeat to 127.3.73.254:35855 (0 consecutive failures): Network error: Failed to ping master at 127.3.73.254:35855: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
I20260807 10:53:43.608129 26465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:43.607743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.195:42463)
0807 10:53:43.607922 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:43.608002 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 10:53:43.609223 26465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:43.608873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.195:51005)
0807 10:53:43.609030 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:43.609107 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260807 10:53:43.609495 26461 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 10:53:43.610129 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 26277
I20260807 10:53:44.097570 26467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:44.096500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.194:47265)
0807 10:53:44.097180 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:44.097284 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":359,"threads_started":1}
I20260807 10:53:44.586253 26468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:44.585442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.193:56649)
0807 10:53:44.585986 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:44.586085 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":356,"threads_started":1}
I20260807 10:53:44.610985 26469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:44.610088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.195:49137)
0807 10:53:44.610597 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:44.610713 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":249,"threads_started":1}
I20260807 10:53:45.100103 26471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:45.099005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.194:36139)
0807 10:53:45.099759 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:45.099888 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":345,"threads_started":1}
I20260807 10:53:45.588456 26474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:45.587413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.193:59407)
0807 10:53:45.588136 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:45.588247 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":273,"threads_started":1}
I20260807 10:53:45.613122 26475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:45.612110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.195:37503)
0807 10:53:45.612795 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:45.612910 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":258,"threads_started":1}
I20260807 10:53:46.102176 26477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:46.101404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.194:33497)
0807 10:53:46.101875 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:46.101962 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":247,"threads_started":1}
I20260807 10:53:46.590560 26478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:46.589664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.193:34013)
0807 10:53:46.590187 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:46.590353 (+ 166us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":277,"threads_started":1}
I20260807 10:53:46.614996 26480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:46.614159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.195:50991)
0807 10:53:46.614663 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:46.614756 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":268,"threads_started":1}
I20260807 10:53:47.105065 26481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:47.103497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.194:60649)
0807 10:53:47.104346 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:47.104518 (+ 172us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":332,"threads_started":1}
I20260807 10:53:47.592608 26483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:47.591716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.193:48459)
0807 10:53:47.592286 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:47.592382 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":270,"threads_started":1}
I20260807 10:53:47.616992 26486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:47.616052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.195:39919)
0807 10:53:47.616649 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:47.616794 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":340,"threads_started":1}
I20260807 10:53:48.107029 26487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:48.106169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.194:38445)
0807 10:53:48.106718 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:48.106824 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":270,"threads_started":1}
I20260807 10:53:48.594321 26488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:48.593571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.193:45383)
0807 10:53:48.594051 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:48.594143 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":241,"threads_started":1}
I20260807 10:53:48.618965 26490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:48.618121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.195:38133)
0807 10:53:48.618639 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:48.618729 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":263,"threads_started":1}
I20260807 10:53:49.109232 26491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:49.108200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.194:60951)
0807 10:53:49.108868 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:49.109030 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":346,"threads_started":1}
I20260807 10:53:49.596446 26493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:49.595405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.193:40581)
0807 10:53:49.596115 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:49.596206 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":393,"threads_started":1}
I20260807 10:53:49.620846 26494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:49.620037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.195:40589)
0807 10:53:49.620506 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:49.620600 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":243,"threads_started":1}
I20260807 10:53:50.111387 26495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:50.110455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.194:51003)
0807 10:53:50.111047 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:50.111156 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":287,"threads_started":1}
I20260807 10:53:50.598254 26496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:50.597470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.193:51927)
0807 10:53:50.597977 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:50.598066 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":260,"threads_started":1}
I20260807 10:53:50.622963 26497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:50.622010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.195:39879)
0807 10:53:50.622650 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:50.622741 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":332,"threads_started":1}
I20260807 10:53:51.113600 26498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:51.112617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.194:58037)
0807 10:53:51.113237 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:51.113339 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":350,"threads_started":1}
I20260807 10:53:51.600793 26499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:51.599465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.193:33645)
0807 10:53:51.600277 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:51.600461 (+ 184us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":405,"threads_started":1}
I20260807 10:53:51.624861 26501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:51.624000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.195:38303)
0807 10:53:51.624487 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:51.624599 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":240,"threads_started":1}
I20260807 10:53:52.115908 26502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:52.114966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.194:42733)
0807 10:53:52.115555 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:52.115657 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":303,"threads_started":1}
I20260807 10:53:52.603046 26503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:52.602077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.193:52615)
0807 10:53:52.602744 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:52.602846 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":376,"threads_started":1}
I20260807 10:53:52.626819 26504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:52.625997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.195:56887)
0807 10:53:52.626539 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:52.626627 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":244,"threads_started":1}
I20260807 10:53:53.118132 26505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:53.117282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.194:43039)
0807 10:53:53.117852 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:53.117951 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":291,"threads_started":1}
I20260807 10:53:53.605572 26506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:53.604375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.193:43573)
0807 10:53:53.605239 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:53.605334 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":751,"thread_start_us":474,"threads_started":1}
I20260807 10:53:53.629194 26508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:53.628260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.195:42879)
0807 10:53:53.628904 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:53.629006 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":331,"threads_started":1}
I20260807 10:53:54.120322 26511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:54.119361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.194:39123)
0807 10:53:54.119968 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:54.120074 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":291,"threads_started":1}
I20260807 10:53:54.607630 26513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:54.606846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.193:41829)
0807 10:53:54.607355 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:54.607449 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":255,"threads_started":1}
I20260807 10:53:54.631104 26514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:54.630281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.195:56375)
0807 10:53:54.630801 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:54.630905 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":275,"threads_started":1}
I20260807 10:53:55.122427 26516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:55.121527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.194:49691)
0807 10:53:55.122091 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:55.122185 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":304,"threads_started":1}
I20260807 10:53:55.609994 26517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:55.608999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.193:44145)
0807 10:53:55.609632 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:55.609780 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":337,"threads_started":1}
I20260807 10:53:55.633061 26518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:55.632162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.195:60903)
0807 10:53:55.632798 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:55.632890 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":354,"threads_started":1}
I20260807 10:53:56.124642 26519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:56.123631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.194:45365)
0807 10:53:56.124241 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:56.124432 (+ 191us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":311,"threads_started":1}
I20260807 10:53:56.611780 26520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:56.610961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.193:57709)
0807 10:53:56.611466 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:56.611556 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":249,"threads_started":1}
I20260807 10:53:56.635135 26521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:56.634183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.195:41381)
0807 10:53:56.634823 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:56.634932 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":368,"threads_started":1}
I20260807 10:53:57.126797 26522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:57.125813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.194:58955)
0807 10:53:57.126455 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:57.126577 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":331,"threads_started":1}
I20260807 10:53:57.614028 26524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:57.613241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.193:34643)
0807 10:53:57.613733 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:57.613823 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":257,"threads_started":1}
I20260807 10:53:57.636976 26526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:57.636141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.195:33489)
0807 10:53:57.636623 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:57.636706 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":262,"threads_started":1}
I20260807 10:53:58.129168 26529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:58.128199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.194:36483)
0807 10:53:58.128833 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:58.128961 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":256,"threads_started":1}
I20260807 10:53:58.614367 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 25813
I20260807 10:53:58.616544 26530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:58.615323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.193:56051)
0807 10:53:58.615976 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:58.616159 (+ 183us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":580,"threads_started":1}
I20260807 10:53:58.637725 26533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:58.615747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49746 (local address 127.3.73.193:44389)
0807 10:53:58.617012 (+ 1265us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:58.617036 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:58.617097 (+ 61us) server_negotiation.cc:407] Connection header received
0807 10:53:58.619220 (+ 2123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:58.619231 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:58.619718 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:58.620128 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:58.622000 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:58.623471 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:58.625251 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:58.626167 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:58.626407 (+ 240us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:58.629834 (+ 3427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:58.629844 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:58.629864 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:58.629922 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:58.633367 (+ 3445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:58.634319 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:58.634330 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:58.634350 (+ 20us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:58.634624 (+ 274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:58.635087 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:58.635093 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:58.635097 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:58.635886 (+ 789us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:58.636275 (+ 389us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:58.636833 (+ 558us) server_negotiation.cc:299] Negotiation successful
0807 10:53:58.637376 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1035,"thread_start_us":827,"threads_started":1}
I20260807 10:53:58.639283 26534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:58.638368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:35855 (local address 127.3.73.195:38971)
0807 10:53:58.638939 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0807 10:53:58.639066 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:35855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":294,"threads_started":1}
I20260807 10:53:58.639704 26060 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:53:58.730860 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 25813
I20260807 10:53:58.763267 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 26071
I20260807 10:53:58.783455 26536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:58.764414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55174 (local address 127.3.73.194:32827)
0807 10:53:58.765024 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:58.765041 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:58.765083 (+ 42us) server_negotiation.cc:407] Connection header received
0807 10:53:58.766766 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:58.766776 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:58.767265 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:58.767598 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:58.769181 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:58.770674 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:58.772040 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:58.772968 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:58.773124 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:58.776073 (+ 2949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:58.776081 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:58.776099 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:58.776148 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:58.779761 (+ 3613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:58.780642 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:58.780651 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:58.780659 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:58.781052 (+ 393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:58.781419 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:58.781423 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:58.781425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:58.781997 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:58.782250 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:58.782754 (+ 504us) server_negotiation.cc:299] Negotiation successful
0807 10:53:58.783115 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":306,"threads_started":1}
I20260807 10:53:58.785140 26263 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:53:58.873353 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 26071
I20260807 10:53:58.907639 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 26277
I20260807 10:53:58.929263 26539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:53:58.908960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42524 (local address 127.3.73.195:42781)
0807 10:53:58.909664 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0807 10:53:58.909680 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 10:53:58.909727 (+ 47us) server_negotiation.cc:407] Connection header received
0807 10:53:58.911415 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:53:58.911427 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:53:58.911919 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:53:58.912237 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:53:58.913933 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:58.915232 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:58.916680 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:53:58.917763 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:53:58.917968 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:53:58.921169 (+ 3201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:53:58.921181 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:53:58.921203 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:53:58.921256 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:53:58.925068 (+ 3812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:58.926236 (+ 1168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:58.926247 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:58.926256 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:58.926484 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:53:58.926994 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:53:58.926998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:53:58.927000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:53:58.927618 (+ 618us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:53:58.927937 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0807 10:53:58.928349 (+ 412us) server_negotiation.cc:299] Negotiation successful
0807 10:53:58.928939 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":338,"threads_started":1}
I20260807 10:53:58.930853 26453 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:53:59.019405 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 26277
2026-08-07T10:53:59Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (26044 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:53:59 dist-test-slave-x5fp krb5kdc[26544](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:53:59 dist-test-slave-x5fp krb5kdc[26544](info): set up 2 sockets
Aug 07 10:53:59 dist-test-slave-x5fp krb5kdc[26544](info): commencing operation
krb5kdc: starting...
W20260807 10:54:00.439154 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.366s user 0.000s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:54:00 dist-test-slave-x5fp krb5kdc[26544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:54:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:54:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.73.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
I20260807 10:54:00.825641 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:34761
--webserver_interface=127.3.73.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:35801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:34761
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:54:01.212608 26560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:54:01.213194 26560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:54:01.213371 26560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:54:01.221222 26560 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 10:54:01.221382 26560 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 10:54:01.221484 26560 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 10:54:01.221585 26560 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:54:01.221692 26560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:54:01.221830 26560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:54:01.221923 26560 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:54:01.222007 26560 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:54:01.232895 26560 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:34761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:34761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:54:01.234436 26560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:54:01.235993 26560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:54:01.249985 26572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:54:01.249962 26568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:54:01.250063 26575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:54:01.250779 26560 server_base.cc:1034] running on GCE node
I20260807 10:54:01.251752 26560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:54:01.253557 26560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:54:01.254842 26560 hybrid_clock.cc:648] HybridClock initialized: now 1786100041254760 us; error 62 us; skew 500 ppm
I20260807 10:54:01.255226 26560 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:54:01 dist-test-slave-x5fp krb5kdc[26544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100041, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:54:01.259425 26560 init.cc:377] Logged in from keytab as kudu/127.3.73.254@KRBTEST.COM (short username kudu)
I20260807 10:54:01.261659 26560 webserver.cc:460] Webserver started at http://127.3.73.254:38125/ using document root <none> and password file <none>
I20260807 10:54:01.262607 26560 fs_manager.cc:359] Metadata directory not provided
I20260807 10:54:01.262775 26560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:54:01.263168 26560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:54:01.267033 26560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "d3884469691e4d2fa1d13d24268c10fb"
format_stamp: "Formatted at 2026-08-07 10:54:01 on dist-test-slave-x5fp"
server_key: "df773170c018a442728ae50840739c8b"
server_key_iv: "ba29fa248fc99913c6384284143d3c36"
server_key_version: "encryptionkey@0"
I20260807 10:54:01.268169 26560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "d3884469691e4d2fa1d13d24268c10fb"
format_stamp: "Formatted at 2026-08-07 10:54:01 on dist-test-slave-x5fp"
server_key: "df773170c018a442728ae50840739c8b"
server_key_iv: "ba29fa248fc99913c6384284143d3c36"
server_key_version: "encryptionkey@0"
I20260807 10:54:01.275099 26560 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260807 10:54:01.279654 26585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:54:01.281572 26560 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 10:54:01.281884 26560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "d3884469691e4d2fa1d13d24268c10fb"
format_stamp: "Formatted at 2026-08-07 10:54:01 on dist-test-slave-x5fp"
server_key: "df773170c018a442728ae50840739c8b"
server_key_iv: "ba29fa248fc99913c6384284143d3c36"
server_key_version: "encryptionkey@0"
I20260807 10:54:01.282187 26560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:54:01.304658 26560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:54:01.310500 26560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:54:01.310866 26560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:54:01.334484 26560 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:34761
I20260807 10:54:01.334534 26666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:34761 every 8 connection(s)
I20260807 10:54:01.336022 26560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:54:01.340803 26667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:54:01.343509 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 26560
I20260807 10:54:01.343817 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:54:01.344655 3367 external_mini_cluster.cc:1428] Setting key f55d1b5aea328e6858a0cf226a59b6a1
I20260807 10:54:01.356432 26667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb: Bootstrap starting.
I20260807 10:54:01.360584 26667 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb: Neither blocks nor log segments found. Creating new log.
I20260807 10:54:01.362169 26667 log.cc:824] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb: Log is configured to *not* fsync() on all Append() calls
Aug 07 10:54:01 dist-test-slave-x5fp krb5kdc[26544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:54:01.366839 26667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb: No bootstrap required, opened a new log
I20260807 10:54:01.375686 26671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:01.347617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49236 (local address 127.3.73.254:34761)
0807 10:54:01.348552 (+ 935us) server_negotiation.cc:206] Beginning negotiation
0807 10:54:01.348570 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 10:54:01.348633 (+ 63us) server_negotiation.cc:407] Connection header received
0807 10:54:01.351304 (+ 2671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:54:01.351342 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:54:01.351989 (+ 647us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:54:01.352544 (+ 555us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:54:01.354235 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:01.356521 (+ 2286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:01.358462 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:01.359587 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:01.359819 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:01.365847 (+ 6028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:54:01.365865 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:54:01.365891 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:54:01.365948 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:54:01.369969 (+ 4021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:54:01.371191 (+ 1222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:54:01.371200 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:54:01.371208 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:54:01.371425 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:54:01.371968 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:54:01.371973 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:54:01.371975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:54:01.372671 (+ 696us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:54:01.373026 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0807 10:54:01.373696 (+ 670us) server_negotiation.cc:299] Negotiation successful
0807 10:54:01.374408 (+ 712us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":394,"threads_started":1}
I20260807 10:54:01.377446 26667 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3884469691e4d2fa1d13d24268c10fb" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 34761 } }
I20260807 10:54:01.377907 26667 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:54:01.378043 26667 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3884469691e4d2fa1d13d24268c10fb, State: Initialized, Role: FOLLOWER
I20260807 10:54:01.378669 26667 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3884469691e4d2fa1d13d24268c10fb" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 34761 } }
I20260807 10:54:01.378993 26667 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:54:01.379175 26667 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:54:01.379379 26667 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:54:01.382977 26667 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3884469691e4d2fa1d13d24268c10fb" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 34761 } }
I20260807 10:54:01.383607 26667 leader_election.cc:302] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3884469691e4d2fa1d13d24268c10fb; no voters:
I20260807 10:54:01.384460 26667 leader_election.cc:288] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:54:01.384609 26673 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:54:01.385975 26673 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [term 1 LEADER]: Becoming Leader. State: Replica: d3884469691e4d2fa1d13d24268c10fb, State: Running, Role: LEADER
I20260807 10:54:01.386749 26673 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3884469691e4d2fa1d13d24268c10fb" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 34761 } }
I20260807 10:54:01.387636 26667 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:54:01.391916 26674 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d3884469691e4d2fa1d13d24268c10fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3884469691e4d2fa1d13d24268c10fb" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 34761 } } }
I20260807 10:54:01.392464 26674 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [sys.catalog]: This master's current role is: LEADER
I20260807 10:54:01.391863 26675 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3884469691e4d2fa1d13d24268c10fb. Latest consensus state: current_term: 1 leader_uuid: "d3884469691e4d2fa1d13d24268c10fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3884469691e4d2fa1d13d24268c10fb" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 34761 } } }
I20260807 10:54:01.393217 26675 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [sys.catalog]: This master's current role is: LEADER
W20260807 10:54:01.403076 26697 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 10:54:01.403286 26697 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 10:54:01.404029 26684 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:54:01.406874 26684 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:54:01.418273 26684 catalog_manager.cc:1347] Generated new cluster ID: a0a7f60e516044948ef21b7d4609ba74
I20260807 10:54:01.418431 26684 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:54:01.473845 26684 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:54:01.475251 26684 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:54:01.484165 26684 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb: Generated new TSK 0
I20260807 10:54:01.485337 26684 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.73.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab.
Entry for principal kudu/127.3.73.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab.
I20260807 10:54:01.590626 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.193:0
--local_ip_for_outbound_sockets=127.3.73.193
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:34761
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:35801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:54:01.984530 26702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:54:01.984946 26702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:54:01.985055 26702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:54:01.993253 26702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:54:01.993397 26702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:54:01.993528 26702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.193
I20260807 10:54:02.004109 26702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:34761
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.193
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:54:02.005560 26702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:54:02.006978 26702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:54:02.021236 26711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:54:02.021263 26713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:54:02.021263 26710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:54:02.022640 26702 server_base.cc:1034] running on GCE node
I20260807 10:54:02.023631 26702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:54:02.024839 26702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:54:02.026118 26702 hybrid_clock.cc:648] HybridClock initialized: now 1786100042026069 us; error 41 us; skew 500 ppm
I20260807 10:54:02.026494 26702 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:54:02 dist-test-slave-x5fp krb5kdc[26544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100042, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:54:02.034253 26702 init.cc:377] Logged in from keytab as kudu/127.3.73.193@KRBTEST.COM (short username kudu)
I20260807 10:54:02.035904 26702 webserver.cc:460] Webserver started at http://127.3.73.193:45709/ using document root <none> and password file <none>
I20260807 10:54:02.036651 26702 fs_manager.cc:359] Metadata directory not provided
I20260807 10:54:02.036864 26702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:54:02.037177 26702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:54:02.040288 26702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/data/instance:
uuid: "a14d469484c94dd7baf344d9330b3fd3"
format_stamp: "Formatted at 2026-08-07 10:54:02 on dist-test-slave-x5fp"
server_key: "ce4aec35ac94456d43fd1e1ee2679a92"
server_key_iv: "6f96675806152327a853aeb9e18c19dc"
server_key_version: "encryptionkey@0"
I20260807 10:54:02.041190 26702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance:
uuid: "a14d469484c94dd7baf344d9330b3fd3"
format_stamp: "Formatted at 2026-08-07 10:54:02 on dist-test-slave-x5fp"
server_key: "ce4aec35ac94456d43fd1e1ee2679a92"
server_key_iv: "6f96675806152327a853aeb9e18c19dc"
server_key_version: "encryptionkey@0"
I20260807 10:54:02.047005 26702 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260807 10:54:02.051529 26724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:54:02.053431 26702 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 10:54:02.053696 26702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/wal
uuid: "a14d469484c94dd7baf344d9330b3fd3"
format_stamp: "Formatted at 2026-08-07 10:54:02 on dist-test-slave-x5fp"
server_key: "ce4aec35ac94456d43fd1e1ee2679a92"
server_key_iv: "6f96675806152327a853aeb9e18c19dc"
server_key_version: "encryptionkey@0"
I20260807 10:54:02.054023 26702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:54:02.074596 26702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:54:02.079787 26702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:54:02.080165 26702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:54:02.081424 26702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:54:02.083178 26702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:54:02.083454 26702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:54:02.083737 26702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:54:02.083861 26702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:54:02.124250 26702 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.193:36585
I20260807 10:54:02.124300 26905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.193:36585 every 8 connection(s)
I20260807 10:54:02.125763 26702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
I20260807 10:54:02.129097 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 26702
I20260807 10:54:02.129350 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance
I20260807 10:54:02.130326 3367 external_mini_cluster.cc:1428] Setting key e460c61f86be6f4769d73434c84db0b8
Aug 07 10:54:02 dist-test-slave-x5fp krb5kdc[26544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100042, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:54:02.150144 26909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:02.128495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:34917 (local address 127.3.73.254:34761)
0807 10:54:02.129088 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0807 10:54:02.129095 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:54:02.130299 (+ 1204us) server_negotiation.cc:407] Connection header received
0807 10:54:02.132507 (+ 2208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:54:02.132513 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:54:02.132612 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:54:02.132772 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:54:02.134735 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:02.135506 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:02.137817 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:02.138298 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:02.138461 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:02.142904 (+ 4443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:54:02.142910 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:54:02.142914 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:54:02.142956 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:54:02.146185 (+ 3229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:54:02.147219 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:54:02.147225 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:54:02.147228 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:54:02.147382 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:54:02.147985 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:54:02.147990 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:54:02.147992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:54:02.148318 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:54:02.148504 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 10:54:02.149443 (+ 939us) server_negotiation.cc:299] Negotiation successful
0807 10:54:02.149722 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":304,"threads_started":1}
I20260807 10:54:02.151540 26910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:02.129046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.193:34917)
0807 10:54:02.130059 (+ 1013us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:02.130120 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 10:54:02.132171 (+ 2051us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:54:02.133215 (+ 1044us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:54:02.133225 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:54:02.133828 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:54:02.134518 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:54:02.134540 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:54:02.135723 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:54:02.135797 (+ 74us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:54:02.137574 (+ 1777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:54:02.137594 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:54:02.138607 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:54:02.138617 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:54:02.138909 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:02.139775 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:54:02.139803 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:54:02.142547 (+ 2744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:54:02.146426 (+ 3879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:54:02.146438 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:54:02.146469 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:54:02.147044 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:54:02.147520 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:54:02.147526 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:54:02.147530 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:54:02.147793 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:54:02.148475 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:54:02.148485 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:54:02.148993 (+ 508us) client_negotiation.cc:769] Sending connection context
0807 10:54:02.149305 (+ 312us) client_negotiation.cc:240] Negotiation successful
0807 10:54:02.149981 (+ 676us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":346,"threads_started":1}
I20260807 10:54:02.153707 26906 heartbeater.cc:347] Connected to a master server at 127.3.73.254:34761
I20260807 10:54:02.154116 26906 heartbeater.cc:464] Registering TS with master...
I20260807 10:54:02.155318 26906 heartbeater.cc:511] Master 127.3.73.254:34761 requested a full tablet report, sending...
I20260807 10:54:02.158025 26613 ts_manager.cc:194] Registered new tserver with Master: a14d469484c94dd7baf344d9330b3fd3 (127.3.73.193:36585)
I20260807 10:54:02.159725 26613 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.193@KRBTEST.COM'} at 127.3.73.193:34917
WARNING: no policy specified for kudu/127.3.73.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab.
Entry for principal kudu/127.3.73.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab.
I20260807 10:54:02.213881 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.194:0
--local_ip_for_outbound_sockets=127.3.73.194
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:34761
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:35801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:54:02.599105 26917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:54:02.599519 26917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:54:02.599706 26917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:54:02.607677 26917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:54:02.607880 26917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:54:02.608088 26917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.194
I20260807 10:54:02.618697 26917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:34761
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.194
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:54:02.620194 26917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:54:02.621788 26917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:54:02.637244 26924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:54:02.637167 26925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:54:02.637215 26927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:54:02.638226 26917 server_base.cc:1034] running on GCE node
I20260807 10:54:02.638912 26917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:54:02.640040 26917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:54:02.641305 26917 hybrid_clock.cc:648] HybridClock initialized: now 1786100042641247 us; error 67 us; skew 500 ppm
I20260807 10:54:02.641685 26917 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:54:02 dist-test-slave-x5fp krb5kdc[26544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100042, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:54:02.645160 26917 init.cc:377] Logged in from keytab as kudu/127.3.73.194@KRBTEST.COM (short username kudu)
I20260807 10:54:02.647075 26917 webserver.cc:460] Webserver started at http://127.3.73.194:38319/ using document root <none> and password file <none>
I20260807 10:54:02.647815 26917 fs_manager.cc:359] Metadata directory not provided
I20260807 10:54:02.647931 26917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:54:02.648202 26917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:54:02.651705 26917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/data/instance:
uuid: "436e4f5a88964f58b970888835c9928c"
format_stamp: "Formatted at 2026-08-07 10:54:02 on dist-test-slave-x5fp"
server_key: "c141ad2f0662218af90dde747a70f1ec"
server_key_iv: "8c7bf8e5a7cd98dbef9db3d5f4a08264"
server_key_version: "encryptionkey@0"
I20260807 10:54:02.653105 26917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance:
uuid: "436e4f5a88964f58b970888835c9928c"
format_stamp: "Formatted at 2026-08-07 10:54:02 on dist-test-slave-x5fp"
server_key: "c141ad2f0662218af90dde747a70f1ec"
server_key_iv: "8c7bf8e5a7cd98dbef9db3d5f4a08264"
server_key_version: "encryptionkey@0"
I20260807 10:54:02.659178 26917 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 10:54:02.663393 26936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:54:02.665074 26917 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 10:54:02.665261 26917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/wal
uuid: "436e4f5a88964f58b970888835c9928c"
format_stamp: "Formatted at 2026-08-07 10:54:02 on dist-test-slave-x5fp"
server_key: "c141ad2f0662218af90dde747a70f1ec"
server_key_iv: "8c7bf8e5a7cd98dbef9db3d5f4a08264"
server_key_version: "encryptionkey@0"
I20260807 10:54:02.665477 26917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:54:02.680857 26917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:54:02.685781 26917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:54:02.686161 26917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:54:02.687474 26917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:54:02.689178 26917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:54:02.689268 26917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:54:02.689365 26917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:54:02.689415 26917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:54:02.726504 26917 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.194:43253
I20260807 10:54:02.726559 27109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.194:43253 every 8 connection(s)
I20260807 10:54:02.727927 26917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
I20260807 10:54:02.733649 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 26917
I20260807 10:54:02.733961 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance
I20260807 10:54:02.734483 3367 external_mini_cluster.cc:1428] Setting key eb6b87052c480ba0d327f45e505adbc6
Aug 07 10:54:02 dist-test-slave-x5fp krb5kdc[26544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100042, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:54:02.750638 27113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:02.730892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:48365 (local address 127.3.73.254:34761)
0807 10:54:02.731388 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0807 10:54:02.731393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:54:02.732333 (+ 940us) server_negotiation.cc:407] Connection header received
0807 10:54:02.734619 (+ 2286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:54:02.734625 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:54:02.734728 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:54:02.734861 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:54:02.737077 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:02.737848 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:02.739510 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:02.739962 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:02.740109 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:02.744067 (+ 3958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:54:02.744074 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:54:02.744078 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:54:02.744118 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:54:02.747051 (+ 2933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:54:02.747981 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:54:02.747986 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:54:02.747989 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:54:02.748130 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:54:02.748632 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:54:02.748637 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:54:02.748639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:54:02.748998 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:54:02.749170 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 10:54:02.749965 (+ 795us) server_negotiation.cc:299] Negotiation successful
0807 10:54:02.750222 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":275,"threads_started":1}
I20260807 10:54:02.751716 27114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:02.731398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.194:48365)
0807 10:54:02.732148 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:02.732195 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 10:54:02.734293 (+ 2098us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:54:02.735179 (+ 886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:54:02.735192 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:54:02.735968 (+ 776us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:54:02.736890 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:54:02.736914 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:54:02.738025 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:54:02.738029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:54:02.739350 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:54:02.739363 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:54:02.740219 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:54:02.740227 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:54:02.740484 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:02.741344 (+ 860us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:54:02.741371 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:54:02.743771 (+ 2400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:54:02.747269 (+ 3498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:54:02.747281 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:54:02.747310 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:54:02.747820 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:54:02.748257 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:54:02.748265 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:54:02.748271 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:54:02.748496 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:54:02.749142 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:54:02.749151 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:54:02.749552 (+ 401us) client_negotiation.cc:769] Sending connection context
0807 10:54:02.749847 (+ 295us) client_negotiation.cc:240] Negotiation successful
0807 10:54:02.750461 (+ 614us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":311,"threads_started":1}
I20260807 10:54:02.753497 27111 heartbeater.cc:347] Connected to a master server at 127.3.73.254:34761
I20260807 10:54:02.753868 27111 heartbeater.cc:464] Registering TS with master...
I20260807 10:54:02.754657 27111 heartbeater.cc:511] Master 127.3.73.254:34761 requested a full tablet report, sending...
I20260807 10:54:02.756263 26613 ts_manager.cc:194] Registered new tserver with Master: 436e4f5a88964f58b970888835c9928c (127.3.73.194:43253)
I20260807 10:54:02.757193 26613 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.194@KRBTEST.COM'} at 127.3.73.194:48365
WARNING: no policy specified for kudu/127.3.73.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab.
Entry for principal kudu/127.3.73.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab.
I20260807 10:54:02.816551 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.195:0
--local_ip_for_outbound_sockets=127.3.73.195
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:34761
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:35801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 10:54:03.164263 26906 heartbeater.cc:503] Master 127.3.73.254:34761 was elected leader, sending a full tablet report...
W20260807 10:54:03.183180 27118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:54:03.183571 27118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:54:03.183759 27118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:54:03.191793 27118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:54:03.191967 27118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:54:03.192160 27118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.195
I20260807 10:54:03.202879 27118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:34761
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.195
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:54:03.204339 27118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:54:03.205809 27118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:54:03.220302 27127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:54:03.220254 27126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:54:03.220363 27129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:54:03.221582 27118 server_base.cc:1034] running on GCE node
I20260807 10:54:03.222548 27118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:54:03.223701 27118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:54:03.224967 27118 hybrid_clock.cc:648] HybridClock initialized: now 1786100043224916 us; error 64 us; skew 500 ppm
I20260807 10:54:03.225364 27118 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:54:03 dist-test-slave-x5fp krb5kdc[26544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100043, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:54:03.229079 27118 init.cc:377] Logged in from keytab as kudu/127.3.73.195@KRBTEST.COM (short username kudu)
I20260807 10:54:03.230829 27118 webserver.cc:460] Webserver started at http://127.3.73.195:34619/ using document root <none> and password file <none>
I20260807 10:54:03.231683 27118 fs_manager.cc:359] Metadata directory not provided
I20260807 10:54:03.231845 27118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:54:03.232179 27118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:54:03.235452 27118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/data/instance:
uuid: "11c27e06913849dd92d8124803c4a913"
format_stamp: "Formatted at 2026-08-07 10:54:03 on dist-test-slave-x5fp"
server_key: "bc9543176defdd68d1cca3a24cc19511"
server_key_iv: "5ddaef1ce08b1eacd0bfe97cd67faac4"
server_key_version: "encryptionkey@0"
I20260807 10:54:03.236327 27118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance:
uuid: "11c27e06913849dd92d8124803c4a913"
format_stamp: "Formatted at 2026-08-07 10:54:03 on dist-test-slave-x5fp"
server_key: "bc9543176defdd68d1cca3a24cc19511"
server_key_iv: "5ddaef1ce08b1eacd0bfe97cd67faac4"
server_key_version: "encryptionkey@0"
I20260807 10:54:03.242550 27118 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260807 10:54:03.246974 27138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:54:03.248920 27118 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 10:54:03.249152 27118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/wal
uuid: "11c27e06913849dd92d8124803c4a913"
format_stamp: "Formatted at 2026-08-07 10:54:03 on dist-test-slave-x5fp"
server_key: "bc9543176defdd68d1cca3a24cc19511"
server_key_iv: "5ddaef1ce08b1eacd0bfe97cd67faac4"
server_key_version: "encryptionkey@0"
I20260807 10:54:03.249423 27118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:54:03.266822 27118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:54:03.273207 27118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:54:03.273574 27118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:54:03.274689 27118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:54:03.276252 27118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:54:03.276366 27118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:54:03.276530 27118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:54:03.276612 27118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:54:03.318449 27118 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.195:46863
I20260807 10:54:03.318563 27308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.195:46863 every 8 connection(s)
I20260807 10:54:03.319972 27118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
I20260807 10:54:03.324180 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 27118
I20260807 10:54:03.324774 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance
I20260807 10:54:03.326068 3367 external_mini_cluster.cc:1428] Setting key 96bf693d47c5f742fbe6898866ebbf3b
Aug 07 10:54:03 dist-test-slave-x5fp krb5kdc[26544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100043, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:54:03.354423 27312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:03.323540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:41259 (local address 127.3.73.254:34761)
0807 10:54:03.333271 (+ 9731us) server_negotiation.cc:206] Beginning negotiation
0807 10:54:03.333280 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:54:03.334052 (+ 772us) server_negotiation.cc:407] Connection header received
0807 10:54:03.337298 (+ 3246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:54:03.337304 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:54:03.337396 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:54:03.337514 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:54:03.339621 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.340469 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.342289 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.342798 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.342956 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:03.347151 (+ 4195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:54:03.347156 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:54:03.347159 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:54:03.347200 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:54:03.350402 (+ 3202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:54:03.351356 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:54:03.351369 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:54:03.351372 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:54:03.351509 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:54:03.352006 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:54:03.352012 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:54:03.352015 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:54:03.352316 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:54:03.352513 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0807 10:54:03.353691 (+ 1178us) server_negotiation.cc:299] Negotiation successful
0807 10:54:03.353973 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9550,"thread_start_us":9376,"threads_started":1}
I20260807 10:54:03.355562 27314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:03.324248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.195:41259)
0807 10:54:03.333769 (+ 9521us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:03.333834 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 10:54:03.336823 (+ 2989us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:54:03.337997 (+ 1174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:54:03.338012 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:54:03.338743 (+ 731us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:54:03.339413 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:54:03.339435 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.340661 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.340667 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:54:03.342112 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:54:03.342128 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.343068 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.343076 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:54:03.343352 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:03.344267 (+ 915us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:54:03.344294 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:54:03.346875 (+ 2581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:54:03.350568 (+ 3693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:54:03.350578 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:54:03.350603 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:54:03.351221 (+ 618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:54:03.351634 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:54:03.351639 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:54:03.351643 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:54:03.351871 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:54:03.352472 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:54:03.352481 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:54:03.353146 (+ 665us) client_negotiation.cc:769] Sending connection context
0807 10:54:03.353560 (+ 414us) client_negotiation.cc:240] Negotiation successful
0807 10:54:03.354189 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9114,"thread_start_us":11083,"threads_started":1}
I20260807 10:54:03.357359 27310 heartbeater.cc:347] Connected to a master server at 127.3.73.254:34761
I20260807 10:54:03.357748 27310 heartbeater.cc:464] Registering TS with master...
I20260807 10:54:03.358592 27310 heartbeater.cc:511] Master 127.3.73.254:34761 requested a full tablet report, sending...
I20260807 10:54:03.360363 26613 ts_manager.cc:194] Registered new tserver with Master: 11c27e06913849dd92d8124803c4a913 (127.3.73.195:46863)
I20260807 10:54:03.361384 26613 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.195@KRBTEST.COM'} at 127.3.73.195:41259
I20260807 10:54:03.364352 3367 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 10:54:03.389889 27312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:03.370070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49252 (local address 127.3.73.254:34761)
0807 10:54:03.370309 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 10:54:03.370320 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 10:54:03.373225 (+ 2905us) server_negotiation.cc:407] Connection header received
0807 10:54:03.373364 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:54:03.373369 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:54:03.373478 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:54:03.373607 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:54:03.375495 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.376309 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.378093 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.378757 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.378931 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:03.382083 (+ 3152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:54:03.382090 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:54:03.382094 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:54:03.382142 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:54:03.385428 (+ 3286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:54:03.386580 (+ 1152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:54:03.386585 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:54:03.386588 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:54:03.386768 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:54:03.387387 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:54:03.387392 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:54:03.387394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:54:03.387881 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:54:03.388139 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0807 10:54:03.388875 (+ 736us) server_negotiation.cc:299] Negotiation successful
0807 10:54:03.389312 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 10:54:03.395533 26613 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49252:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 10:54:03.404305 26613 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 10:54:03.438869 27335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:03.420744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:43253 (local address 127.0.0.1:34570)
0807 10:54:03.426923 (+ 6179us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:03.426946 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 10:54:03.427335 (+ 389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:54:03.430402 (+ 3067us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:54:03.430407 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:54:03.430432 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:54:03.431045 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:54:03.431064 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.432848 (+ 1784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.432857 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:54:03.435653 (+ 2796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:54:03.435667 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.437714 (+ 2047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.437721 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:54:03.437927 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:03.437984 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 10:54:03.438126 (+ 142us) client_negotiation.cc:240] Negotiation successful
0807 10:54:03.438377 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6004,"mutex_wait_us":440,"thread_start_us":5364,"threads_started":1}
I20260807 10:54:03.439339 27333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:03.420748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:46863 (local address 127.0.0.1:36146)
0807 10:54:03.425609 (+ 4861us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:03.425669 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 10:54:03.426129 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:54:03.429518 (+ 3389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:54:03.429531 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:54:03.429585 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:54:03.430115 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:54:03.430131 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.433030 (+ 2899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.433035 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:54:03.436339 (+ 3304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:54:03.436357 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.438303 (+ 1946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.438310 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:54:03.438471 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:03.438519 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 10:54:03.438589 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 10:54:03.438736 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4543,"mutex_wait_us":115,"thread_start_us":4309,"threads_started":1}
I20260807 10:54:03.439538 27334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:03.420967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34570 (local address 127.3.73.194:43253)
0807 10:54:03.429382 (+ 8415us) server_negotiation.cc:206] Beginning negotiation
0807 10:54:03.429393 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 10:54:03.429424 (+ 31us) server_negotiation.cc:407] Connection header received
0807 10:54:03.429542 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:54:03.429553 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:54:03.429853 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:54:03.430093 (+ 240us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:54:03.431325 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.432643 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.435845 (+ 3202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.437206 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.437356 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:03.438572 (+ 1216us) server_negotiation.cc:1035] Waiting for connection context
0807 10:54:03.438667 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 10:54:03.438898 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8268,"thread_start_us":8127,"threads_started":1}
I20260807 10:54:03.440330 27338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:03.428942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:36585 (local address 127.0.0.1:60326)
0807 10:54:03.429875 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:03.429901 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 10:54:03.430073 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:54:03.431043 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:54:03.431050 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:54:03.431077 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:54:03.431596 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:54:03.431612 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.434134 (+ 2522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.434146 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:54:03.436677 (+ 2531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:54:03.436690 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.439357 (+ 2667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.439368 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:54:03.439708 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:03.439752 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 10:54:03.439872 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 10:54:03.440004 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":775,"mutex_wait_us":63,"thread_start_us":597,"threads_started":1}
I20260807 10:54:03.440513 27337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:03.426835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60326 (local address 127.3.73.193:36585)
0807 10:54:03.428901 (+ 2066us) server_negotiation.cc:206] Beginning negotiation
0807 10:54:03.428915 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 10:54:03.430241 (+ 1326us) server_negotiation.cc:407] Connection header received
0807 10:54:03.430367 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:54:03.430374 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:54:03.430630 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:54:03.430850 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:54:03.431820 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.433859 (+ 2039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.436927 (+ 3068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.438562 (+ 1635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.438712 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:03.438960 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 10:54:03.439956 (+ 996us) server_negotiation.cc:299] Negotiation successful
0807 10:54:03.440176 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1786,"thread_start_us":1634,"threads_started":1}
I20260807 10:54:03.442022 27336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:03.421378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36146 (local address 127.3.73.195:46863)
0807 10:54:03.428242 (+ 6864us) server_negotiation.cc:206] Beginning negotiation
0807 10:54:03.428258 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 10:54:03.428536 (+ 278us) server_negotiation.cc:407] Connection header received
0807 10:54:03.428682 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:54:03.428692 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:54:03.429055 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:54:03.430370 (+ 1315us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:54:03.430739 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.432855 (+ 2116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.436539 (+ 3684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.438067 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.439563 (+ 1496us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:03.441219 (+ 1656us) server_negotiation.cc:1035] Waiting for connection context
0807 10:54:03.441324 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 10:54:03.441573 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6587,"thread_start_us":6368,"threads_started":1}
I20260807 10:54:03.444213 27014 tablet_service.cc:1467] Processing CreateTablet for tablet f8e96408ee4143e7b095cf7ff8a24724 (DEFAULT_TABLE table=test-table [id=f44fa625c3084530b4e65db1ed84a09d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:54:03.445649 27014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8e96408ee4143e7b095cf7ff8a24724. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:54:03.444983 26798 tablet_service.cc:1467] Processing CreateTablet for tablet f8e96408ee4143e7b095cf7ff8a24724 (DEFAULT_TABLE table=test-table [id=f44fa625c3084530b4e65db1ed84a09d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:54:03.446753 26798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8e96408ee4143e7b095cf7ff8a24724. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:54:03.446295 27213 tablet_service.cc:1467] Processing CreateTablet for tablet f8e96408ee4143e7b095cf7ff8a24724 (DEFAULT_TABLE table=test-table [id=f44fa625c3084530b4e65db1ed84a09d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:54:03.447903 27213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8e96408ee4143e7b095cf7ff8a24724. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:54:03.464813 27339 tablet_bootstrap.cc:492] T f8e96408ee4143e7b095cf7ff8a24724 P 436e4f5a88964f58b970888835c9928c: Bootstrap starting.
I20260807 10:54:03.466909 27342 tablet_bootstrap.cc:492] T f8e96408ee4143e7b095cf7ff8a24724 P a14d469484c94dd7baf344d9330b3fd3: Bootstrap starting.
I20260807 10:54:03.468451 27340 tablet_bootstrap.cc:492] T f8e96408ee4143e7b095cf7ff8a24724 P 11c27e06913849dd92d8124803c4a913: Bootstrap starting.
I20260807 10:54:03.470494 27339 tablet_bootstrap.cc:654] T f8e96408ee4143e7b095cf7ff8a24724 P 436e4f5a88964f58b970888835c9928c: Neither blocks nor log segments found. Creating new log.
I20260807 10:54:03.472661 27339 log.cc:824] T f8e96408ee4143e7b095cf7ff8a24724 P 436e4f5a88964f58b970888835c9928c: Log is configured to *not* fsync() on all Append() calls
I20260807 10:54:03.473886 27342 tablet_bootstrap.cc:654] T f8e96408ee4143e7b095cf7ff8a24724 P a14d469484c94dd7baf344d9330b3fd3: Neither blocks nor log segments found. Creating new log.
I20260807 10:54:03.473892 27340 tablet_bootstrap.cc:654] T f8e96408ee4143e7b095cf7ff8a24724 P 11c27e06913849dd92d8124803c4a913: Neither blocks nor log segments found. Creating new log.
I20260807 10:54:03.476018 27340 log.cc:824] T f8e96408ee4143e7b095cf7ff8a24724 P 11c27e06913849dd92d8124803c4a913: Log is configured to *not* fsync() on all Append() calls
I20260807 10:54:03.475817 27342 log.cc:824] T f8e96408ee4143e7b095cf7ff8a24724 P a14d469484c94dd7baf344d9330b3fd3: Log is configured to *not* fsync() on all Append() calls
I20260807 10:54:03.477465 27339 tablet_bootstrap.cc:492] T f8e96408ee4143e7b095cf7ff8a24724 P 436e4f5a88964f58b970888835c9928c: No bootstrap required, opened a new log
I20260807 10:54:03.477928 27339 ts_tablet_manager.cc:1397] T f8e96408ee4143e7b095cf7ff8a24724 P 436e4f5a88964f58b970888835c9928c: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.004s
I20260807 10:54:03.480695 27342 tablet_bootstrap.cc:492] T f8e96408ee4143e7b095cf7ff8a24724 P a14d469484c94dd7baf344d9330b3fd3: No bootstrap required, opened a new log
I20260807 10:54:03.480695 27340 tablet_bootstrap.cc:492] T f8e96408ee4143e7b095cf7ff8a24724 P 11c27e06913849dd92d8124803c4a913: No bootstrap required, opened a new log
I20260807 10:54:03.481069 27342 ts_tablet_manager.cc:1397] T f8e96408ee4143e7b095cf7ff8a24724 P a14d469484c94dd7baf344d9330b3fd3: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.005s
I20260807 10:54:03.481068 27340 ts_tablet_manager.cc:1397] T f8e96408ee4143e7b095cf7ff8a24724 P 11c27e06913849dd92d8124803c4a913: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.003s
I20260807 10:54:03.489099 27339 raft_consensus.cc:348] T f8e96408ee4143e7b095cf7ff8a24724 P 436e4f5a88964f58b970888835c9928c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "436e4f5a88964f58b970888835c9928c" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43253 } } peers { permanent_uuid: "11c27e06913849dd92d8124803c4a913" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 46863 } } peers { permanent_uuid: "a14d469484c94dd7baf344d9330b3fd3" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 36585 } }
I20260807 10:54:03.489639 27339 raft_consensus.cc:374] T f8e96408ee4143e7b095cf7ff8a24724 P 436e4f5a88964f58b970888835c9928c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:54:03.489755 27339 raft_consensus.cc:729] T f8e96408ee4143e7b095cf7ff8a24724 P 436e4f5a88964f58b970888835c9928c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 436e4f5a88964f58b970888835c9928c, State: Initialized, Role: FOLLOWER
I20260807 10:54:03.490528 27339 consensus_queue.cc:260] T f8e96408ee4143e7b095cf7ff8a24724 P 436e4f5a88964f58b970888835c9928c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "436e4f5a88964f58b970888835c9928c" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43253 } } peers { permanent_uuid: "11c27e06913849dd92d8124803c4a913" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 46863 } } peers { permanent_uuid: "a14d469484c94dd7baf344d9330b3fd3" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 36585 } }
I20260807 10:54:03.491942 27111 heartbeater.cc:503] Master 127.3.73.254:34761 was elected leader, sending a full tablet report...
I20260807 10:54:03.492892 27339 ts_tablet_manager.cc:1428] T f8e96408ee4143e7b095cf7ff8a24724 P 436e4f5a88964f58b970888835c9928c: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260807 10:54:03.493172 27340 raft_consensus.cc:348] T f8e96408ee4143e7b095cf7ff8a24724 P 11c27e06913849dd92d8124803c4a913 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "436e4f5a88964f58b970888835c9928c" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43253 } } peers { permanent_uuid: "11c27e06913849dd92d8124803c4a913" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 46863 } } peers { permanent_uuid: "a14d469484c94dd7baf344d9330b3fd3" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 36585 } }
I20260807 10:54:03.493172 27342 raft_consensus.cc:348] T f8e96408ee4143e7b095cf7ff8a24724 P a14d469484c94dd7baf344d9330b3fd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "436e4f5a88964f58b970888835c9928c" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43253 } } peers { permanent_uuid: "11c27e06913849dd92d8124803c4a913" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 46863 } } peers { permanent_uuid: "a14d469484c94dd7baf344d9330b3fd3" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 36585 } }
I20260807 10:54:03.493702 27342 raft_consensus.cc:374] T f8e96408ee4143e7b095cf7ff8a24724 P a14d469484c94dd7baf344d9330b3fd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:54:03.493708 27340 raft_consensus.cc:374] T f8e96408ee4143e7b095cf7ff8a24724 P 11c27e06913849dd92d8124803c4a913 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:54:03.493827 27340 raft_consensus.cc:729] T f8e96408ee4143e7b095cf7ff8a24724 P 11c27e06913849dd92d8124803c4a913 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11c27e06913849dd92d8124803c4a913, State: Initialized, Role: FOLLOWER
I20260807 10:54:03.493860 27342 raft_consensus.cc:729] T f8e96408ee4143e7b095cf7ff8a24724 P a14d469484c94dd7baf344d9330b3fd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a14d469484c94dd7baf344d9330b3fd3, State: Initialized, Role: FOLLOWER
I20260807 10:54:03.494616 27342 consensus_queue.cc:260] T f8e96408ee4143e7b095cf7ff8a24724 P a14d469484c94dd7baf344d9330b3fd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "436e4f5a88964f58b970888835c9928c" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43253 } } peers { permanent_uuid: "11c27e06913849dd92d8124803c4a913" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 46863 } } peers { permanent_uuid: "a14d469484c94dd7baf344d9330b3fd3" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 36585 } }
I20260807 10:54:03.494585 27340 consensus_queue.cc:260] T f8e96408ee4143e7b095cf7ff8a24724 P 11c27e06913849dd92d8124803c4a913 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "436e4f5a88964f58b970888835c9928c" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43253 } } peers { permanent_uuid: "11c27e06913849dd92d8124803c4a913" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 46863 } } peers { permanent_uuid: "a14d469484c94dd7baf344d9330b3fd3" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 36585 } }
I20260807 10:54:03.497169 27340 ts_tablet_manager.cc:1428] T f8e96408ee4143e7b095cf7ff8a24724 P 11c27e06913849dd92d8124803c4a913: Time spent starting tablet: real 0.016s user 0.015s sys 0.001s
I20260807 10:54:03.497148 27342 ts_tablet_manager.cc:1428] T f8e96408ee4143e7b095cf7ff8a24724 P a14d469484c94dd7baf344d9330b3fd3: Time spent starting tablet: real 0.016s user 0.011s sys 0.004s
I20260807 10:54:03.498255 27310 heartbeater.cc:503] Master 127.3.73.254:34761 was elected leader, sending a full tablet report...
I20260807 10:54:03.516108 27349 raft_consensus.cc:482] T f8e96408ee4143e7b095cf7ff8a24724 P a14d469484c94dd7baf344d9330b3fd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 10:54:03.516449 27349 raft_consensus.cc:504] T f8e96408ee4143e7b095cf7ff8a24724 P a14d469484c94dd7baf344d9330b3fd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "436e4f5a88964f58b970888835c9928c" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43253 } } peers { permanent_uuid: "11c27e06913849dd92d8124803c4a913" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 46863 } } peers { permanent_uuid: "a14d469484c94dd7baf344d9330b3fd3" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 36585 } }
I20260807 10:54:03.518424 27349 leader_election.cc:288] T f8e96408ee4143e7b095cf7ff8a24724 P a14d469484c94dd7baf344d9330b3fd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 436e4f5a88964f58b970888835c9928c (127.3.73.194:43253), 11c27e06913849dd92d8124803c4a913 (127.3.73.195:46863)
I20260807 10:54:03.520795 27347 raft_consensus.cc:482] T f8e96408ee4143e7b095cf7ff8a24724 P 436e4f5a88964f58b970888835c9928c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 10:54:03.521167 27347 raft_consensus.cc:504] T f8e96408ee4143e7b095cf7ff8a24724 P 436e4f5a88964f58b970888835c9928c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "436e4f5a88964f58b970888835c9928c" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43253 } } peers { permanent_uuid: "11c27e06913849dd92d8124803c4a913" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 46863 } } peers { permanent_uuid: "a14d469484c94dd7baf344d9330b3fd3" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 36585 } }
I20260807 10:54:03.523391 27347 leader_election.cc:288] T f8e96408ee4143e7b095cf7ff8a24724 P 436e4f5a88964f58b970888835c9928c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 11c27e06913849dd92d8124803c4a913 (127.3.73.195:46863), a14d469484c94dd7baf344d9330b3fd3 (127.3.73.193:36585)
I20260807 10:54:03.529429 27336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:03.519044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:36959 (local address 127.3.73.195:46863)
0807 10:54:03.519240 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 10:54:03.519247 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:54:03.519820 (+ 573us) server_negotiation.cc:407] Connection header received
0807 10:54:03.520085 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:54:03.520089 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:54:03.520150 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:54:03.520251 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:54:03.521060 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.522155 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.525077 (+ 2922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.526947 (+ 1870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.527384 (+ 437us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:03.527536 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 10:54:03.528697 (+ 1161us) server_negotiation.cc:299] Negotiation successful
0807 10:54:03.528968 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 10:54:03.529762 27334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:03.518958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:59863 (local address 127.3.73.194:43253)
0807 10:54:03.519175 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 10:54:03.519182 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:54:03.520342 (+ 1160us) server_negotiation.cc:407] Connection header received
0807 10:54:03.520762 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:54:03.520778 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:54:03.520846 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:54:03.521068 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:54:03.521815 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.522766 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.525963 (+ 3197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.527898 (+ 1935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.528080 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:03.528189 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 10:54:03.529127 (+ 938us) server_negotiation.cc:299] Negotiation successful
0807 10:54:03.529349 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 10:54:03.530825 27352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:03.518735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:46863 (local address 127.3.73.193:36959)
0807 10:54:03.519688 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:03.519705 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 10:54:03.519918 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:54:03.520272 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:54:03.520277 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:54:03.520305 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:54:03.520796 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:54:03.520812 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.522318 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.522325 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:54:03.524818 (+ 2493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:54:03.524843 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.527967 (+ 3124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.527980 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:54:03.528303 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:03.528367 (+ 64us) client_negotiation.cc:769] Sending connection context
0807 10:54:03.529906 (+ 1539us) client_negotiation.cc:240] Negotiation successful
0807 10:54:03.530161 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":837,"thread_start_us":1145,"threads_started":1}
I20260807 10:54:03.531021 27353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:03.518776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:43253 (local address 127.3.73.193:59863)
0807 10:54:03.520210 (+ 1434us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:03.520229 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 10:54:03.520458 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:54:03.521168 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:54:03.521172 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:54:03.521203 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:54:03.521622 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:54:03.521634 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.522919 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.522924 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:54:03.525664 (+ 2740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:54:03.525699 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.528534 (+ 2835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.528544 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:54:03.528859 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:03.528914 (+ 55us) client_negotiation.cc:769] Sending connection context
0807 10:54:03.530103 (+ 1189us) client_negotiation.cc:240] Negotiation successful
0807 10:54:03.530329 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1343,"spinlock_wait_cycles":20736,"thread_start_us":614,"threads_started":1}
I20260807 10:54:03.532328 27041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8e96408ee4143e7b095cf7ff8a24724" candidate_uuid: "a14d469484c94dd7baf344d9330b3fd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "436e4f5a88964f58b970888835c9928c" is_pre_election: true
I20260807 10:54:03.532507 27245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8e96408ee4143e7b095cf7ff8a24724" candidate_uuid: "a14d469484c94dd7baf344d9330b3fd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11c27e06913849dd92d8124803c4a913" is_pre_election: true
I20260807 10:54:03.532984 27041 raft_consensus.cc:2454] T f8e96408ee4143e7b095cf7ff8a24724 P 436e4f5a88964f58b970888835c9928c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a14d469484c94dd7baf344d9330b3fd3 in term 0.
I20260807 10:54:03.533187 27245 raft_consensus.cc:2454] T f8e96408ee4143e7b095cf7ff8a24724 P 11c27e06913849dd92d8124803c4a913 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a14d469484c94dd7baf344d9330b3fd3 in term 0.
I20260807 10:54:03.533861 26727 leader_election.cc:302] T f8e96408ee4143e7b095cf7ff8a24724 P a14d469484c94dd7baf344d9330b3fd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 436e4f5a88964f58b970888835c9928c, a14d469484c94dd7baf344d9330b3fd3; no voters:
I20260807 10:54:03.534998 27349 raft_consensus.cc:2781] T f8e96408ee4143e7b095cf7ff8a24724 P a14d469484c94dd7baf344d9330b3fd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 10:54:03.535308 27349 raft_consensus.cc:482] T f8e96408ee4143e7b095cf7ff8a24724 P a14d469484c94dd7baf344d9330b3fd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 10:54:03.535496 27349 raft_consensus.cc:3037] T f8e96408ee4143e7b095cf7ff8a24724 P a14d469484c94dd7baf344d9330b3fd3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:54:03.539412 27349 raft_consensus.cc:504] T f8e96408ee4143e7b095cf7ff8a24724 P a14d469484c94dd7baf344d9330b3fd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "436e4f5a88964f58b970888835c9928c" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43253 } } peers { permanent_uuid: "11c27e06913849dd92d8124803c4a913" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 46863 } } peers { permanent_uuid: "a14d469484c94dd7baf344d9330b3fd3" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 36585 } }
I20260807 10:54:03.541131 27245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8e96408ee4143e7b095cf7ff8a24724" candidate_uuid: "a14d469484c94dd7baf344d9330b3fd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11c27e06913849dd92d8124803c4a913"
I20260807 10:54:03.541144 27041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8e96408ee4143e7b095cf7ff8a24724" candidate_uuid: "a14d469484c94dd7baf344d9330b3fd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "436e4f5a88964f58b970888835c9928c"
I20260807 10:54:03.541409 27245 raft_consensus.cc:3037] T f8e96408ee4143e7b095cf7ff8a24724 P 11c27e06913849dd92d8124803c4a913 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:54:03.541442 27041 raft_consensus.cc:3037] T f8e96408ee4143e7b095cf7ff8a24724 P 436e4f5a88964f58b970888835c9928c [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:54:03.542233 27356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:03.523590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:46863 (local address 127.3.73.194:35925)
0807 10:54:03.532481 (+ 8891us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:03.532506 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 10:54:03.532774 (+ 268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:54:03.533386 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:54:03.533390 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:54:03.533419 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:54:03.533877 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:54:03.533894 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.536220 (+ 2326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.536225 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:54:03.538852 (+ 2627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:54:03.538874 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.540947 (+ 2073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.540954 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:54:03.541188 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:03.541224 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 10:54:03.541369 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 10:54:03.541536 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8682,"thread_start_us":8734,"threads_started":1}
I20260807 10:54:03.543042 27336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:03.523870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:35925 (local address 127.3.73.195:46863)
0807 10:54:03.529769 (+ 5899us) server_negotiation.cc:206] Beginning negotiation
0807 10:54:03.529775 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:54:03.532619 (+ 2844us) server_negotiation.cc:407] Connection header received
0807 10:54:03.532953 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:54:03.532960 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:54:03.533073 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:54:03.533155 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:54:03.534061 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.535546 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.539143 (+ 3597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.540456 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.540634 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:03.540760 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 10:54:03.542519 (+ 1759us) server_negotiation.cc:299] Negotiation successful
0807 10:54:03.542681 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5784,"thread_start_us":8350,"threads_started":1}
I20260807 10:54:03.544487 27349 leader_election.cc:288] T f8e96408ee4143e7b095cf7ff8a24724 P a14d469484c94dd7baf344d9330b3fd3 [CANDIDATE]: Term 1 election: Requested vote from peers 436e4f5a88964f58b970888835c9928c (127.3.73.194:43253), 11c27e06913849dd92d8124803c4a913 (127.3.73.195:46863)
I20260807 10:54:03.544323 27244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8e96408ee4143e7b095cf7ff8a24724" candidate_uuid: "436e4f5a88964f58b970888835c9928c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11c27e06913849dd92d8124803c4a913" is_pre_election: true
I20260807 10:54:03.545900 27245 raft_consensus.cc:2454] T f8e96408ee4143e7b095cf7ff8a24724 P 11c27e06913849dd92d8124803c4a913 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a14d469484c94dd7baf344d9330b3fd3 in term 1.
I20260807 10:54:03.546372 26726 leader_election.cc:302] T f8e96408ee4143e7b095cf7ff8a24724 P a14d469484c94dd7baf344d9330b3fd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11c27e06913849dd92d8124803c4a913, a14d469484c94dd7baf344d9330b3fd3; no voters:
I20260807 10:54:03.546770 27349 raft_consensus.cc:2781] T f8e96408ee4143e7b095cf7ff8a24724 P a14d469484c94dd7baf344d9330b3fd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:54:03.547003 27041 raft_consensus.cc:2454] T f8e96408ee4143e7b095cf7ff8a24724 P 436e4f5a88964f58b970888835c9928c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a14d469484c94dd7baf344d9330b3fd3 in term 1.
I20260807 10:54:03.547197 27337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:03.534338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:60569 (local address 127.3.73.193:36585)
0807 10:54:03.534856 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0807 10:54:03.534869 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 10:54:03.539730 (+ 4861us) server_negotiation.cc:407] Connection header received
0807 10:54:03.539921 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:54:03.539929 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:54:03.540026 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:54:03.540139 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:54:03.540998 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.542083 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.544773 (+ 2690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.546045 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.546199 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:03.546279 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 10:54:03.546798 (+ 519us) server_negotiation.cc:299] Negotiation successful
0807 10:54:03.546939 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 10:54:03.547293 27360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:03.533050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:36585 (local address 127.3.73.194:60569)
0807 10:54:03.539578 (+ 6528us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:03.539601 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 10:54:03.539772 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:54:03.540216 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:54:03.540221 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:54:03.540252 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:54:03.540693 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:54:03.540708 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.542291 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.542303 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:54:03.544490 (+ 2187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:54:03.544519 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.546382 (+ 1863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.546392 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:54:03.546581 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:03.546621 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 10:54:03.546755 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 10:54:03.546904 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6375,"thread_start_us":6144,"threads_started":1}
I20260807 10:54:03.547864 27349 raft_consensus.cc:686] T f8e96408ee4143e7b095cf7ff8a24724 P a14d469484c94dd7baf344d9330b3fd3 [term 1 LEADER]: Becoming Leader. State: Replica: a14d469484c94dd7baf344d9330b3fd3, State: Running, Role: LEADER
I20260807 10:54:03.548903 26830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8e96408ee4143e7b095cf7ff8a24724" candidate_uuid: "436e4f5a88964f58b970888835c9928c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a14d469484c94dd7baf344d9330b3fd3" is_pre_election: true
I20260807 10:54:03.549084 27349 consensus_queue.cc:237] T f8e96408ee4143e7b095cf7ff8a24724 P a14d469484c94dd7baf344d9330b3fd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "436e4f5a88964f58b970888835c9928c" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43253 } } peers { permanent_uuid: "11c27e06913849dd92d8124803c4a913" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 46863 } } peers { permanent_uuid: "a14d469484c94dd7baf344d9330b3fd3" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 36585 } }
I20260807 10:54:03.550303 26937 leader_election.cc:302] T f8e96408ee4143e7b095cf7ff8a24724 P 436e4f5a88964f58b970888835c9928c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 436e4f5a88964f58b970888835c9928c; no voters: 11c27e06913849dd92d8124803c4a913, a14d469484c94dd7baf344d9330b3fd3
I20260807 10:54:03.550787 27347 raft_consensus.cc:2726] T f8e96408ee4143e7b095cf7ff8a24724 P 436e4f5a88964f58b970888835c9928c [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 10:54:03.555073 26612 catalog_manager.cc:5495] T f8e96408ee4143e7b095cf7ff8a24724 P a14d469484c94dd7baf344d9330b3fd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a14d469484c94dd7baf344d9330b3fd3 (127.3.73.193). New cstate: current_term: 1 leader_uuid: "a14d469484c94dd7baf344d9330b3fd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "436e4f5a88964f58b970888835c9928c" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11c27e06913849dd92d8124803c4a913" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 46863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a14d469484c94dd7baf344d9330b3fd3" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 36585 } health_report { overall_health: HEALTHY } } }
W20260807 10:54:03.572409 27311 tablet.cc:2369] T f8e96408ee4143e7b095cf7ff8a24724 P 11c27e06913849dd92d8124803c4a913: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:54:03.578155 27337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:03.571855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60332 (local address 127.3.73.193:36585)
0807 10:54:03.572042 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 10:54:03.572047 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:54:03.572067 (+ 20us) server_negotiation.cc:407] Connection header received
0807 10:54:03.572327 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:54:03.572332 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:54:03.572464 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:54:03.572667 (+ 203us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:54:03.573279 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.574102 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.575614 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.576106 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.576215 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:03.576835 (+ 620us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:54:03.577514 (+ 679us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:54:03.577638 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 10:54:03.577771 (+ 133us) server_negotiation.cc:299] Negotiation successful
0807 10:54:03.577891 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 10:54:03.588142 27245 raft_consensus.cc:1261] T f8e96408ee4143e7b095cf7ff8a24724 P 11c27e06913849dd92d8124803c4a913 [term 1 FOLLOWER]: Refusing update from remote peer a14d469484c94dd7baf344d9330b3fd3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:54:03.588232 27041 raft_consensus.cc:1261] T f8e96408ee4143e7b095cf7ff8a24724 P 436e4f5a88964f58b970888835c9928c [term 1 FOLLOWER]: Refusing update from remote peer a14d469484c94dd7baf344d9330b3fd3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:54:03.589382 27363 consensus_queue.cc:1035] T f8e96408ee4143e7b095cf7ff8a24724 P a14d469484c94dd7baf344d9330b3fd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "436e4f5a88964f58b970888835c9928c" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:54:03.589792 27349 consensus_queue.cc:1035] T f8e96408ee4143e7b095cf7ff8a24724 P a14d469484c94dd7baf344d9330b3fd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11c27e06913849dd92d8124803c4a913" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 46863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:54:03.604103 27368 mvcc.cc:205] Tried to move back new op lower bound from 7315865778524995584 to 7315865778382434304. Current Snapshot: MvccSnapshot[applied={T|T < 7315865778524995584}]
I20260807 10:54:03.621429 26613 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:54:03.621795 26613 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:54:03.626286 26613 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb: Sending DeleteTablet for 3 replicas of tablet f8e96408ee4143e7b095cf7ff8a24724
I20260807 10:54:03.627889 27014 tablet_service.cc:1514] Processing DeleteTablet for tablet f8e96408ee4143e7b095cf7ff8a24724 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:54:03 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:34570
I20260807 10:54:03.627956 26798 tablet_service.cc:1514] Processing DeleteTablet for tablet f8e96408ee4143e7b095cf7ff8a24724 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:54:03 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:60326
I20260807 10:54:03.628273 27213 tablet_service.cc:1514] Processing DeleteTablet for tablet f8e96408ee4143e7b095cf7ff8a24724 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:54:03 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:36146
I20260807 10:54:03.628984 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 26702
I20260807 10:54:03.640676 27382 tablet_replica.cc:330] T f8e96408ee4143e7b095cf7ff8a24724 P a14d469484c94dd7baf344d9330b3fd3: stopping tablet replica
I20260807 10:54:03.641337 27386 tablet_replica.cc:330] T f8e96408ee4143e7b095cf7ff8a24724 P 11c27e06913849dd92d8124803c4a913: stopping tablet replica
I20260807 10:54:03.641649 27381 tablet_replica.cc:330] T f8e96408ee4143e7b095cf7ff8a24724 P 436e4f5a88964f58b970888835c9928c: stopping tablet replica
I20260807 10:54:03.641964 27386 raft_consensus.cc:2229] T f8e96408ee4143e7b095cf7ff8a24724 P 11c27e06913849dd92d8124803c4a913 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:54:03.642196 27381 raft_consensus.cc:2229] T f8e96408ee4143e7b095cf7ff8a24724 P 436e4f5a88964f58b970888835c9928c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:54:03.642695 27386 raft_consensus.cc:2258] T f8e96408ee4143e7b095cf7ff8a24724 P 11c27e06913849dd92d8124803c4a913 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:54:03.642863 27382 raft_consensus.cc:2229] T f8e96408ee4143e7b095cf7ff8a24724 P a14d469484c94dd7baf344d9330b3fd3 [term 1 LEADER]: Raft consensus shutting down.
I20260807 10:54:03.643491 27381 raft_consensus.cc:2258] T f8e96408ee4143e7b095cf7ff8a24724 P 436e4f5a88964f58b970888835c9928c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:54:03.644446 27382 raft_consensus.cc:2258] T f8e96408ee4143e7b095cf7ff8a24724 P a14d469484c94dd7baf344d9330b3fd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:54:03.647209 27381 ts_tablet_manager.cc:1905] T f8e96408ee4143e7b095cf7ff8a24724 P 436e4f5a88964f58b970888835c9928c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:54:03.647218 27382 ts_tablet_manager.cc:1905] T f8e96408ee4143e7b095cf7ff8a24724 P a14d469484c94dd7baf344d9330b3fd3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:54:03.648072 27386 ts_tablet_manager.cc:1905] T f8e96408ee4143e7b095cf7ff8a24724 P 11c27e06913849dd92d8124803c4a913: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 10:54:03 dist-test-slave-x5fp krb5kdc[26544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.193@KRBTEST.COM
I20260807 10:54:03.660914 27382 ts_tablet_manager.cc:1918] T f8e96408ee4143e7b095cf7ff8a24724 P a14d469484c94dd7baf344d9330b3fd3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:54:03.661181 27382 log.cc:1196] T f8e96408ee4143e7b095cf7ff8a24724 P a14d469484c94dd7baf344d9330b3fd3: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/wal/wals/f8e96408ee4143e7b095cf7ff8a24724
I20260807 10:54:03.662014 27382 ts_tablet_manager.cc:1939] T f8e96408ee4143e7b095cf7ff8a24724 P a14d469484c94dd7baf344d9330b3fd3: Deleting consensus metadata
I20260807 10:54:03.663040 27386 ts_tablet_manager.cc:1918] T f8e96408ee4143e7b095cf7ff8a24724 P 11c27e06913849dd92d8124803c4a913: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:54:03.663321 27386 log.cc:1196] T f8e96408ee4143e7b095cf7ff8a24724 P 11c27e06913849dd92d8124803c4a913: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/wal/wals/f8e96408ee4143e7b095cf7ff8a24724
I20260807 10:54:03.663532 27381 ts_tablet_manager.cc:1918] T f8e96408ee4143e7b095cf7ff8a24724 P 436e4f5a88964f58b970888835c9928c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:54:03.663802 27381 log.cc:1196] T f8e96408ee4143e7b095cf7ff8a24724 P 436e4f5a88964f58b970888835c9928c: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/wal/wals/f8e96408ee4143e7b095cf7ff8a24724
I20260807 10:54:03.664038 27386 ts_tablet_manager.cc:1939] T f8e96408ee4143e7b095cf7ff8a24724 P 11c27e06913849dd92d8124803c4a913: Deleting consensus metadata
I20260807 10:54:03.664247 26588 catalog_manager.cc:4841] TS a14d469484c94dd7baf344d9330b3fd3 (127.3.73.193:36585): tablet f8e96408ee4143e7b095cf7ff8a24724 (table test-table [id=f44fa625c3084530b4e65db1ed84a09d]) successfully deleted
I20260807 10:54:03.664645 27381 ts_tablet_manager.cc:1939] T f8e96408ee4143e7b095cf7ff8a24724 P 436e4f5a88964f58b970888835c9928c: Deleting consensus metadata
I20260807 10:54:03.666412 26588 catalog_manager.cc:4841] TS 11c27e06913849dd92d8124803c4a913 (127.3.73.195:46863): tablet f8e96408ee4143e7b095cf7ff8a24724 (table test-table [id=f44fa625c3084530b4e65db1ed84a09d]) successfully deleted
I20260807 10:54:03.666697 26590 catalog_manager.cc:4841] TS 436e4f5a88964f58b970888835c9928c (127.3.73.194:43253): tablet f8e96408ee4143e7b095cf7ff8a24724 (table test-table [id=f44fa625c3084530b4e65db1ed84a09d]) successfully deleted
I20260807 10:54:03.668635 27337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:03.631160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60334 (local address 127.3.73.193:36585)
0807 10:54:03.631415 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0807 10:54:03.631424 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:54:03.641950 (+ 10526us) server_negotiation.cc:407] Connection header received
0807 10:54:03.642313 (+ 363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:54:03.642320 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:54:03.642420 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:54:03.642501 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:54:03.644683 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.645951 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.648221 (+ 2270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.649050 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.649194 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:03.656244 (+ 7050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:54:03.656262 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:54:03.656285 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:54:03.656352 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:54:03.663736 (+ 7384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:54:03.665791 (+ 2055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:54:03.665802 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:54:03.665806 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:54:03.666004 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:54:03.666606 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:54:03.666611 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:54:03.666615 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:54:03.667216 (+ 601us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:54:03.667610 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0807 10:54:03.667956 (+ 346us) server_negotiation.cc:299] Negotiation successful
0807 10:54:03.668246 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 10:54:03.669438 26901 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:54:03.775355 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 26702
I20260807 10:54:03.806890 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 26917
Aug 07 10:54:03 dist-test-slave-x5fp krb5kdc[26544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.194@KRBTEST.COM
I20260807 10:54:03.824946 27334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:03.807795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34574 (local address 127.3.73.194:43253)
0807 10:54:03.807995 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 10:54:03.808000 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:54:03.808020 (+ 20us) server_negotiation.cc:407] Connection header received
0807 10:54:03.808114 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:54:03.808118 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:54:03.808204 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:54:03.808301 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:54:03.809788 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.810550 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.811933 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.812374 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.812491 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:03.817912 (+ 5421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:54:03.817930 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:54:03.817947 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:54:03.818000 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:54:03.821601 (+ 3601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:54:03.822539 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:54:03.822547 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:54:03.822550 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:54:03.822702 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:54:03.823152 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:54:03.823155 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:54:03.823157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:54:03.823590 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:54:03.823863 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0807 10:54:03.824262 (+ 399us) server_negotiation.cc:299] Negotiation successful
0807 10:54:03.824462 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 10:54:03.825742 27102 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:54:03.924124 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 26917
I20260807 10:54:03.953261 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 27118
Aug 07 10:54:03 dist-test-slave-x5fp krb5kdc[26544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100040, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.195@KRBTEST.COM
I20260807 10:54:03.974061 27336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:03.954496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36162 (local address 127.3.73.195:46863)
0807 10:54:03.954859 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0807 10:54:03.954870 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 10:54:03.955360 (+ 490us) server_negotiation.cc:407] Connection header received
0807 10:54:03.955458 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:54:03.955465 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:54:03.955589 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:54:03.955734 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:54:03.958247 (+ 2513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.959111 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.960793 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:03.961429 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:03.961594 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:03.967237 (+ 5643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:54:03.967264 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:54:03.967282 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:54:03.967338 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:54:03.970846 (+ 3508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:54:03.971756 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:54:03.971771 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:54:03.971773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:54:03.971917 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:54:03.972318 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:54:03.972323 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:54:03.972326 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:54:03.972833 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:54:03.973150 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0807 10:54:03.973481 (+ 331us) server_negotiation.cc:299] Negotiation successful
0807 10:54:03.973726 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140}
I20260807 10:54:03.974848 27303 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:54:04.071465 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 27118
I20260807 10:54:04.101094 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 26560
I20260807 10:54:04.102361 26659 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:54:04.219619 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 26560
I20260807 10:54:24.254755 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:34761
--webserver_interface=127.3.73.254
--webserver_port=38125
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:35801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:34761
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:54:24.632539 27397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:54:24.632936 27397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:54:24.633150 27397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:54:24.643358 27397 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 10:54:24.643535 27397 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 10:54:24.643608 27397 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 10:54:24.643693 27397 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:54:24.643774 27397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:54:24.643885 27397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:54:24.643977 27397 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:54:24.644078 27397 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:54:24.656867 27397 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:34761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:34761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=38125
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:54:24.658223 27397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:54:24.659675 27397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:54:24.688783 27407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:54:24.688894 27408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:54:24.688855 27410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:54:24.690114 27397 server_base.cc:1034] running on GCE node
I20260807 10:54:24.691514 27397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:54:24.693390 27397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:54:24.694643 27397 hybrid_clock.cc:648] HybridClock initialized: now 1786100064694567 us; error 40 us; skew 500 ppm
I20260807 10:54:24.695107 27397 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:54:24 dist-test-slave-x5fp krb5kdc[26544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100064, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:54:24.702936 27397 init.cc:377] Logged in from keytab as kudu/127.3.73.254@KRBTEST.COM (short username kudu)
I20260807 10:54:24.704962 27397 webserver.cc:460] Webserver started at http://127.3.73.254:38125/ using document root <none> and password file <none>
I20260807 10:54:24.705878 27397 fs_manager.cc:359] Metadata directory not provided
I20260807 10:54:24.706058 27397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:54:24.713042 27397 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 10:54:24.716440 27424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:54:24.718199 27397 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 10:54:24.718449 27397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "d3884469691e4d2fa1d13d24268c10fb"
format_stamp: "Formatted at 2026-08-07 10:54:01 on dist-test-slave-x5fp"
server_key: "df773170c018a442728ae50840739c8b"
server_key_iv: "ba29fa248fc99913c6384284143d3c36"
server_key_version: "encryptionkey@0"
I20260807 10:54:24.719686 27397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:54:24.736959 27397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:54:24.742098 27397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:54:24.742462 27397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:54:24.763993 27397 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:34761
I20260807 10:54:24.764037 27496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:34761 every 8 connection(s)
I20260807 10:54:24.765719 27397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:54:24.773512 27497 sys_catalog.cc:263] Verifying existing consensus state
I20260807 10:54:24.773522 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 27397
I20260807 10:54:24.774463 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.193:36585
--local_ip_for_outbound_sockets=127.3.73.193
--tserver_master_addrs=127.3.73.254:34761
--webserver_port=45709
--webserver_interface=127.3.73.193
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:35801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 10:54:24.777222 27497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb: Bootstrap starting.
I20260807 10:54:24.798256 27497 log.cc:824] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb: Log is configured to *not* fsync() on all Append() calls
I20260807 10:54:24.811538 27497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 10:54:24.812146 27497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb: Bootstrap complete.
I20260807 10:54:24.822184 27497 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3884469691e4d2fa1d13d24268c10fb" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 34761 } }
I20260807 10:54:24.823175 27497 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3884469691e4d2fa1d13d24268c10fb, State: Initialized, Role: FOLLOWER
I20260807 10:54:24.823889 27497 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3884469691e4d2fa1d13d24268c10fb" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 34761 } }
I20260807 10:54:24.824158 27497 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:54:24.824321 27497 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:54:24.824545 27497 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [term 1 FOLLOWER]: Advancing to term 2
I20260807 10:54:24.828881 27497 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3884469691e4d2fa1d13d24268c10fb" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 34761 } }
I20260807 10:54:24.829443 27497 leader_election.cc:302] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3884469691e4d2fa1d13d24268c10fb; no voters:
I20260807 10:54:24.830160 27497 leader_election.cc:288] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 10:54:24.830324 27501 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [term 2 FOLLOWER]: Leader election won for term 2
I20260807 10:54:24.831647 27501 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [term 2 LEADER]: Becoming Leader. State: Replica: d3884469691e4d2fa1d13d24268c10fb, State: Running, Role: LEADER
I20260807 10:54:24.832258 27501 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3884469691e4d2fa1d13d24268c10fb" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 34761 } }
I20260807 10:54:24.833422 27497 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:54:24.835180 27504 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3884469691e4d2fa1d13d24268c10fb. Latest consensus state: current_term: 2 leader_uuid: "d3884469691e4d2fa1d13d24268c10fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3884469691e4d2fa1d13d24268c10fb" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 34761 } } }
I20260807 10:54:24.835847 27504 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [sys.catalog]: This master's current role is: LEADER
I20260807 10:54:24.835446 27502 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d3884469691e4d2fa1d13d24268c10fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3884469691e4d2fa1d13d24268c10fb" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 34761 } } }
I20260807 10:54:24.837141 27502 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [sys.catalog]: This master's current role is: LEADER
I20260807 10:54:24.843546 27508 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:54:24.852561 27508 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:54:24.854069 27508 catalog_manager.cc:1259] Loaded cluster ID: a0a7f60e516044948ef21b7d4609ba74
I20260807 10:54:24.854183 27508 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:54:24.858286 27508 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:54:24.859866 27508 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb: Loaded TSK: 0
I20260807 10:54:24.905440 27508 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb: Generated new TSK 1
I20260807 10:54:24.906030 27508 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 10:54:25.183477 27499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:54:25.183830 27499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:54:25.184016 27499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:54:25.191788 27499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:54:25.191967 27499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:54:25.192150 27499 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.193
I20260807 10:54:25.202420 27499 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.193:36585
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.73.193
--webserver_port=45709
--tserver_master_addrs=127.3.73.254:34761
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.193
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:54:25.203824 27499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:54:25.205292 27499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:54:25.220844 27533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:54:25.221057 27535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:54:25.221628 27531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:54:25.222342 27499 server_base.cc:1034] running on GCE node
I20260807 10:54:25.223135 27499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:54:25.224269 27499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:54:25.225507 27499 hybrid_clock.cc:648] HybridClock initialized: now 1786100065225426 us; error 66 us; skew 500 ppm
I20260807 10:54:25.225844 27499 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:54:25 dist-test-slave-x5fp krb5kdc[26544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100065, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:54:25.229329 27499 init.cc:377] Logged in from keytab as kudu/127.3.73.193@KRBTEST.COM (short username kudu)
I20260807 10:54:25.231012 27499 webserver.cc:460] Webserver started at http://127.3.73.193:45709/ using document root <none> and password file <none>
I20260807 10:54:25.231760 27499 fs_manager.cc:359] Metadata directory not provided
I20260807 10:54:25.231916 27499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:54:25.238302 27499 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 10:54:25.241437 27544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:54:25.242940 27499 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 10:54:25.243120 27499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/wal
uuid: "a14d469484c94dd7baf344d9330b3fd3"
format_stamp: "Formatted at 2026-08-07 10:54:02 on dist-test-slave-x5fp"
server_key: "ce4aec35ac94456d43fd1e1ee2679a92"
server_key_iv: "6f96675806152327a853aeb9e18c19dc"
server_key_version: "encryptionkey@0"
I20260807 10:54:25.244164 27499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:54:25.260022 27499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:54:25.264372 27499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:54:25.264683 27499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:54:25.265913 27499 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:54:25.267443 27499 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:54:25.267534 27499 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:54:25.267694 27499 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:54:25.267798 27499 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:54:25.303083 27499 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.193:36585
I20260807 10:54:25.303179 27721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.193:36585 every 8 connection(s)
I20260807 10:54:25.304605 27499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
I20260807 10:54:25.310418 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 27499
I20260807 10:54:25.311292 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.194:43253
--local_ip_for_outbound_sockets=127.3.73.194
--tserver_master_addrs=127.3.73.254:34761
--webserver_port=38319
--webserver_interface=127.3.73.194
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:35801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 10:54:25 dist-test-slave-x5fp krb5kdc[26544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100065, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:54:25.330873 27727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:25.307894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.193:50035)
0807 10:54:25.308803 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:25.308850 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 10:54:25.310676 (+ 1826us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:54:25.312320 (+ 1644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:54:25.312327 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:54:25.312962 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:54:25.313507 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:54:25.313526 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:54:25.315967 (+ 2441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:54:25.315971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:54:25.317296 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:54:25.317310 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:54:25.318812 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:54:25.318817 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:54:25.319011 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:25.319751 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:54:25.319774 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:54:25.322075 (+ 2301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:54:25.326207 (+ 4132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:54:25.326218 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:54:25.326233 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:54:25.326691 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:54:25.327178 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:54:25.327183 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:54:25.327186 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:54:25.327387 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:54:25.328190 (+ 803us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:54:25.328198 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:54:25.328556 (+ 358us) client_negotiation.cc:769] Sending connection context
0807 10:54:25.328887 (+ 331us) client_negotiation.cc:240] Negotiation successful
0807 10:54:25.329490 (+ 603us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":352,"threads_started":1}
I20260807 10:54:25.330871 27726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:25.307739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:50035 (local address 127.3.73.254:34761)
0807 10:54:25.308371 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0807 10:54:25.308383 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 10:54:25.308988 (+ 605us) server_negotiation.cc:407] Connection header received
0807 10:54:25.311306 (+ 2318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:54:25.311329 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:54:25.311792 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:54:25.312187 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:54:25.313770 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:25.315776 (+ 2006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:25.317450 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:25.318581 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:25.318792 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:25.322403 (+ 3611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:54:25.322416 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:54:25.322438 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:54:25.322498 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:54:25.326047 (+ 3549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:54:25.326848 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:54:25.326857 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:54:25.326864 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:54:25.327052 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:54:25.327528 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:54:25.327533 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:54:25.327536 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:54:25.328038 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:54:25.328263 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 10:54:25.329132 (+ 869us) server_negotiation.cc:299] Negotiation successful
0807 10:54:25.329541 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":295,"threads_started":1}
I20260807 10:54:25.334336 27723 heartbeater.cc:347] Connected to a master server at 127.3.73.254:34761
I20260807 10:54:25.334693 27723 heartbeater.cc:464] Registering TS with master...
I20260807 10:54:25.335665 27723 heartbeater.cc:511] Master 127.3.73.254:34761 requested a full tablet report, sending...
I20260807 10:54:25.338622 27447 ts_manager.cc:194] Registered new tserver with Master: a14d469484c94dd7baf344d9330b3fd3 (127.3.73.193:36585)
I20260807 10:54:25.341178 27447 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.193@KRBTEST.COM'} at 127.3.73.193:50035
W20260807 10:54:25.677893 27728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:54:25.678241 27728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:54:25.678354 27728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:54:25.685673 27728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:54:25.685776 27728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:54:25.685894 27728 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.194
I20260807 10:54:25.695662 27728 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.194:43253
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.73.194
--webserver_port=38319
--tserver_master_addrs=127.3.73.254:34761
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.194
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:54:25.697227 27728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:54:25.698654 27728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:54:25.712627 27734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:54:25.712592 27735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:54:25.713904 27737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:54:25.714208 27728 server_base.cc:1034] running on GCE node
I20260807 10:54:25.714944 27728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:54:25.716061 27728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:54:25.717348 27728 hybrid_clock.cc:648] HybridClock initialized: now 1786100065717257 us; error 73 us; skew 500 ppm
I20260807 10:54:25.717772 27728 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:54:25 dist-test-slave-x5fp krb5kdc[26544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100065, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:54:25.721637 27728 init.cc:377] Logged in from keytab as kudu/127.3.73.194@KRBTEST.COM (short username kudu)
I20260807 10:54:25.723291 27728 webserver.cc:460] Webserver started at http://127.3.73.194:38319/ using document root <none> and password file <none>
I20260807 10:54:25.724102 27728 fs_manager.cc:359] Metadata directory not provided
I20260807 10:54:25.724272 27728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:54:25.730307 27728 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260807 10:54:25.734084 27748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:54:25.736136 27728 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 10:54:25.736554 27728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/wal
uuid: "436e4f5a88964f58b970888835c9928c"
format_stamp: "Formatted at 2026-08-07 10:54:02 on dist-test-slave-x5fp"
server_key: "c141ad2f0662218af90dde747a70f1ec"
server_key_iv: "8c7bf8e5a7cd98dbef9db3d5f4a08264"
server_key_version: "encryptionkey@0"
I20260807 10:54:25.737990 27728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:54:25.762262 27728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:54:25.767021 27728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:54:25.767302 27728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:54:25.768462 27728 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:54:25.769942 27728 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:54:25.770077 27728 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:54:25.770219 27728 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:54:25.770310 27728 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:54:25.809325 27728 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.194:43253
I20260807 10:54:25.809420 27933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.194:43253 every 8 connection(s)
I20260807 10:54:25.810843 27728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
I20260807 10:54:25.812897 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 27728
I20260807 10:54:25.813711 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.195:46863
--local_ip_for_outbound_sockets=127.3.73.195
--tserver_master_addrs=127.3.73.254:34761
--webserver_port=34619
--webserver_interface=127.3.73.195
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:35801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 10:54:25 dist-test-slave-x5fp krb5kdc[26544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100065, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:54:25.832907 27726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:25.813771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:34225 (local address 127.3.73.254:34761)
0807 10:54:25.814016 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 10:54:25.814025 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:54:25.815305 (+ 1280us) server_negotiation.cc:407] Connection header received
0807 10:54:25.817294 (+ 1989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:54:25.817300 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:54:25.817417 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:54:25.817533 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:54:25.819083 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:25.819796 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:25.821500 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:25.821942 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:25.822078 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:25.825974 (+ 3896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:54:25.825983 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:54:25.825990 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:54:25.826037 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:54:25.829274 (+ 3237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:54:25.830283 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:54:25.830288 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:54:25.830291 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:54:25.830428 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:54:25.830938 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:54:25.830942 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:54:25.830944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:54:25.831246 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:54:25.831380 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 10:54:25.832334 (+ 954us) server_negotiation.cc:299] Negotiation successful
0807 10:54:25.832548 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 10:54:25.834470 27939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:25.814353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.194:34225)
0807 10:54:25.815131 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:25.815178 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 10:54:25.817009 (+ 1831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:54:25.817785 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:54:25.817794 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:54:25.818300 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:54:25.818894 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:54:25.818912 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:54:25.820020 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:54:25.820024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:54:25.821345 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:54:25.821357 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:54:25.822200 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:54:25.822208 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:54:25.822453 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:25.823172 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:54:25.823200 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:54:25.825723 (+ 2523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:54:25.829422 (+ 3699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:54:25.829434 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:54:25.829454 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:54:25.830120 (+ 666us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:54:25.830547 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:54:25.830555 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:54:25.830562 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:54:25.830803 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:54:25.831386 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:54:25.831397 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:54:25.831880 (+ 483us) client_negotiation.cc:769] Sending connection context
0807 10:54:25.832246 (+ 366us) client_negotiation.cc:240] Negotiation successful
0807 10:54:25.832921 (+ 675us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":320,"threads_started":1}
I20260807 10:54:25.836110 27935 heartbeater.cc:347] Connected to a master server at 127.3.73.254:34761
I20260807 10:54:25.836440 27935 heartbeater.cc:464] Registering TS with master...
I20260807 10:54:25.837252 27935 heartbeater.cc:511] Master 127.3.73.254:34761 requested a full tablet report, sending...
I20260807 10:54:25.838761 27447 ts_manager.cc:194] Registered new tserver with Master: 436e4f5a88964f58b970888835c9928c (127.3.73.194:43253)
I20260807 10:54:25.839577 27447 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.194@KRBTEST.COM'} at 127.3.73.194:34225
W20260807 10:54:26.188133 27943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:54:26.188490 27943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:54:26.188668 27943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:54:26.196166 27943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:54:26.196337 27943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:54:26.196548 27943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.195
I20260807 10:54:26.206590 27943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.195:46863
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.73.195
--webserver_port=34619
--tserver_master_addrs=127.3.73.254:34761
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.195
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:54:26.207962 27943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:54:26.209367 27943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:54:26.222967 27955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:54:26.223048 27953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:54:26.223089 27952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:54:26.223919 27943 server_base.cc:1034] running on GCE node
I20260807 10:54:26.224539 27943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:54:26.225558 27943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:54:26.226765 27943 hybrid_clock.cc:648] HybridClock initialized: now 1786100066226726 us; error 30 us; skew 500 ppm
I20260807 10:54:26.227113 27943 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:54:26 dist-test-slave-x5fp krb5kdc[26544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100066, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:54:26.230330 27943 init.cc:377] Logged in from keytab as kudu/127.3.73.195@KRBTEST.COM (short username kudu)
I20260807 10:54:26.231812 27943 webserver.cc:460] Webserver started at http://127.3.73.195:34619/ using document root <none> and password file <none>
I20260807 10:54:26.232492 27943 fs_manager.cc:359] Metadata directory not provided
I20260807 10:54:26.232606 27943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:54:26.238338 27943 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 10:54:26.241185 27965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:54:26.242745 27943 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 10:54:26.242944 27943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/wal
uuid: "11c27e06913849dd92d8124803c4a913"
format_stamp: "Formatted at 2026-08-07 10:54:03 on dist-test-slave-x5fp"
server_key: "bc9543176defdd68d1cca3a24cc19511"
server_key_iv: "5ddaef1ce08b1eacd0bfe97cd67faac4"
server_key_version: "encryptionkey@0"
I20260807 10:54:26.244010 27943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:54:26.291772 27943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:54:26.296356 27943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:54:26.296660 27943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:54:26.297740 27943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:54:26.299155 27943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:54:26.299275 27943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:54:26.299407 27943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:54:26.299508 27943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:54:26.336407 27943 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.195:46863
I20260807 10:54:26.336506 28153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.195:46863 every 8 connection(s)
I20260807 10:54:26.338001 27943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
I20260807 10:54:26.343999 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 27943
I20260807 10:54:26.346109 27723 heartbeater.cc:503] Master 127.3.73.254:34761 was elected leader, sending a full tablet report...
Aug 07 10:54:26 dist-test-slave-x5fp krb5kdc[26544](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100066, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:54:26.363435 28156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:26.340701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:47071 (local address 127.3.73.254:34761)
0807 10:54:26.341193 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0807 10:54:26.341198 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:54:26.342266 (+ 1068us) server_negotiation.cc:407] Connection header received
0807 10:54:26.344362 (+ 2096us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:54:26.344368 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:54:26.344481 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:54:26.344623 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:54:26.347511 (+ 2888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:26.348816 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:26.351238 (+ 2422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:26.351958 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:26.352120 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:26.356486 (+ 4366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:54:26.356495 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:54:26.356501 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:54:26.356549 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:54:26.359988 (+ 3439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:54:26.360878 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:54:26.360883 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:54:26.360888 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:54:26.361016 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:54:26.361524 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:54:26.361529 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:54:26.361531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:54:26.361814 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:54:26.361990 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 10:54:26.362765 (+ 775us) server_negotiation.cc:299] Negotiation successful
0807 10:54:26.363018 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":230,"threads_started":1}
I20260807 10:54:26.364264 28160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:26.345268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45724 (local address 127.3.73.254:34761)
0807 10:54:26.345800 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0807 10:54:26.345804 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 10:54:26.348235 (+ 2431us) server_negotiation.cc:407] Connection header received
0807 10:54:26.348301 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:54:26.348311 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:54:26.348415 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:54:26.348568 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:54:26.350836 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:26.352011 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:26.354055 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:26.354491 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:26.354593 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:26.358119 (+ 3526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:54:26.358124 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:54:26.358130 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:54:26.358183 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:54:26.361266 (+ 3083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:54:26.362042 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:54:26.362054 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:54:26.362057 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:54:26.362172 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:54:26.362578 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:54:26.362583 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:54:26.362588 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:54:26.362919 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:54:26.363270 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0807 10:54:26.363638 (+ 368us) server_negotiation.cc:299] Negotiation successful
0807 10:54:26.363873 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":423,"thread_start_us":294,"threads_started":1}
I20260807 10:54:26.365002 28159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:26.341298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.195:47071)
0807 10:54:26.342094 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:26.342133 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 10:54:26.344014 (+ 1881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:54:26.345005 (+ 991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:54:26.345019 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:54:26.346069 (+ 1050us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:54:26.347159 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:54:26.347194 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:54:26.349013 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:54:26.349019 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:54:26.351085 (+ 2066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:54:26.351099 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:54:26.352243 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:54:26.352251 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:54:26.352611 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:26.353463 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:54:26.353493 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:54:26.356222 (+ 2729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:54:26.360179 (+ 3957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:54:26.360190 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:54:26.360214 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:54:26.360677 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:54:26.361154 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:54:26.361159 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:54:26.361163 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:54:26.361376 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:54:26.361964 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:54:26.361973 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:54:26.362366 (+ 393us) client_negotiation.cc:769] Sending connection context
0807 10:54:26.362660 (+ 294us) client_negotiation.cc:240] Negotiation successful
0807 10:54:26.363373 (+ 713us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":298,"threads_started":1}
I20260807 10:54:26.366611 28154 heartbeater.cc:347] Connected to a master server at 127.3.73.254:34761
I20260807 10:54:26.366989 28154 heartbeater.cc:464] Registering TS with master...
I20260807 10:54:26.367812 28154 heartbeater.cc:511] Master 127.3.73.254:34761 requested a full tablet report, sending...
I20260807 10:54:26.369465 27446 ts_manager.cc:194] Registered new tserver with Master: 11c27e06913849dd92d8124803c4a913 (127.3.73.195:46863)
I20260807 10:54:26.370409 27446 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.195@KRBTEST.COM'} at 127.3.73.195:47071
I20260807 10:54:26.380033 3367 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 10:54:26.387282 28160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:26.381056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45734 (local address 127.3.73.254:34761)
0807 10:54:26.381253 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 10:54:26.381258 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:54:26.381840 (+ 582us) server_negotiation.cc:407] Connection header received
0807 10:54:26.382049 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:54:26.382052 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:54:26.382208 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:54:26.382339 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:54:26.382803 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:26.383544 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:26.385217 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:26.385648 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:26.385752 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:26.386323 (+ 571us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:54:26.386660 (+ 337us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 10:54:26.387027 (+ 367us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45734: token expired
Metrics: {"server-negotiator.queue_time_us":64}
W20260807 10:54:26.387470 28160 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45734: token expired
W20260807 10:54:26.388115 27317 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.3.73.254:34761)
I20260807 10:54:26.402464 28160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:26.388896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45746 (local address 127.3.73.254:34761)
0807 10:54:26.389078 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 10:54:26.389083 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:54:26.389100 (+ 17us) server_negotiation.cc:407] Connection header received
0807 10:54:26.389151 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:54:26.389154 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:54:26.389215 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:54:26.389314 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:54:26.390784 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:26.391552 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:26.393143 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:26.393550 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:26.393655 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:26.396467 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:54:26.396478 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:54:26.396482 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:54:26.396523 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:54:26.399572 (+ 3049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:54:26.400363 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:54:26.400367 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:54:26.400369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:54:26.400510 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:54:26.401067 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:54:26.401071 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:54:26.401073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:54:26.401455 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:54:26.401645 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 10:54:26.402018 (+ 373us) server_negotiation.cc:299] Negotiation successful
0807 10:54:26.402223 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 10:54:26.406308 27446 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45746:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 10:54:26.413935 27446 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 10:54:26.438675 28172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:26.426581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:43253 (local address 127.0.0.1:34976)
0807 10:54:26.428462 (+ 1881us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:26.428494 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0807 10:54:26.428776 (+ 282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:54:26.430064 (+ 1288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:54:26.430072 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:54:26.430112 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:54:26.430574 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:54:26.430583 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:54:26.432530 (+ 1947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:54:26.432536 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:54:26.435083 (+ 2547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:54:26.435101 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:54:26.437540 (+ 2439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:54:26.437548 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:54:26.437759 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:26.437825 (+ 66us) client_negotiation.cc:769] Sending connection context
0807 10:54:26.437983 (+ 158us) client_negotiation.cc:240] Negotiation successful
0807 10:54:26.438213 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1675,"mutex_wait_us":314,"thread_start_us":1516,"threads_started":1}
I20260807 10:54:26.439180 28173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:26.427201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42626 (local address 127.3.73.193:36585)
0807 10:54:26.427861 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0807 10:54:26.427867 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:54:26.428666 (+ 799us) server_negotiation.cc:407] Connection header received
0807 10:54:26.429054 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:54:26.429063 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:54:26.429329 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:54:26.429541 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:54:26.430788 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:26.432321 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:26.435493 (+ 3172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:26.436993 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:26.437106 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:26.437829 (+ 723us) server_negotiation.cc:1035] Waiting for connection context
0807 10:54:26.438452 (+ 623us) server_negotiation.cc:299] Negotiation successful
0807 10:54:26.438637 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":424,"threads_started":1}
I20260807 10:54:26.439464 28170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:26.426681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34976 (local address 127.3.73.194:43253)
0807 10:54:26.427295 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0807 10:54:26.427302 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:54:26.428613 (+ 1311us) server_negotiation.cc:407] Connection header received
0807 10:54:26.429079 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:54:26.429086 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:54:26.429593 (+ 507us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:54:26.429812 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:54:26.430761 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:26.432332 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:26.435319 (+ 2987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:26.436994 (+ 1675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:26.437142 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:26.437359 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 10:54:26.438970 (+ 1611us) server_negotiation.cc:299] Negotiation successful
0807 10:54:26.439163 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":771,"threads_started":1}
I20260807 10:54:26.441197 28171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:26.426610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:36585 (local address 127.0.0.1:42626)
0807 10:54:26.428535 (+ 1925us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:26.428555 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 10:54:26.428796 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:54:26.429882 (+ 1086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:54:26.429889 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:54:26.429918 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:54:26.430549 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:54:26.430566 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:54:26.432539 (+ 1973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:54:26.432544 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:54:26.435273 (+ 2729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:54:26.435289 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:54:26.438041 (+ 2752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:54:26.438048 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:54:26.438219 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:26.438252 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 10:54:26.439519 (+ 1267us) client_negotiation.cc:240] Negotiation successful
0807 10:54:26.440682 (+ 1163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1801,"mutex_wait_us":170,"thread_start_us":2157,"threads_started":1}
I20260807 10:54:26.441236 28174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:26.429158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:46863 (local address 127.0.0.1:60336)
0807 10:54:26.429752 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:26.429771 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 10:54:26.429875 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:54:26.431094 (+ 1219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:54:26.431098 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:54:26.431136 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:54:26.431515 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:54:26.431526 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:54:26.433640 (+ 2114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:54:26.433646 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:54:26.437399 (+ 3753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:54:26.437416 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:54:26.440173 (+ 2757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:54:26.440179 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:54:26.440368 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:26.440399 (+ 31us) client_negotiation.cc:769] Sending connection context
0807 10:54:26.440469 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 10:54:26.440702 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"mutex_wait_us":333,"thread_start_us":722,"threads_started":1}
I20260807 10:54:26.441854 28175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:26.429696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60336 (local address 127.3.73.195:46863)
0807 10:54:26.430480 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0807 10:54:26.430489 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:54:26.430524 (+ 35us) server_negotiation.cc:407] Connection header received
0807 10:54:26.430620 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:54:26.430630 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:54:26.430862 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:54:26.431091 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:54:26.431768 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:26.433428 (+ 1660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:26.437620 (+ 4192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:26.439498 (+ 1878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:26.440892 (+ 1394us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:26.441151 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0807 10:54:26.441233 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 10:54:26.441424 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":417,"threads_started":1}
I20260807 10:54:26.443791 27831 tablet_service.cc:1467] Processing CreateTablet for tablet e39ab77df304447db021abaea59c6c7d (DEFAULT_TABLE table=test-table [id=0fc475f12f504114836c5b430e10888d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:54:26.445335 28040 tablet_service.cc:1467] Processing CreateTablet for tablet e39ab77df304447db021abaea59c6c7d (DEFAULT_TABLE table=test-table [id=0fc475f12f504114836c5b430e10888d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:54:26.445334 27626 tablet_service.cc:1467] Processing CreateTablet for tablet e39ab77df304447db021abaea59c6c7d (DEFAULT_TABLE table=test-table [id=0fc475f12f504114836c5b430e10888d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:54:26.445952 27831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e39ab77df304447db021abaea59c6c7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:54:26.446820 28040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e39ab77df304447db021abaea59c6c7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:54:26.446844 27626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e39ab77df304447db021abaea59c6c7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:54:26.460013 28176 tablet_bootstrap.cc:492] T e39ab77df304447db021abaea59c6c7d P 436e4f5a88964f58b970888835c9928c: Bootstrap starting.
I20260807 10:54:26.463572 28176 tablet_bootstrap.cc:654] T e39ab77df304447db021abaea59c6c7d P 436e4f5a88964f58b970888835c9928c: Neither blocks nor log segments found. Creating new log.
I20260807 10:54:26.464054 28178 tablet_bootstrap.cc:492] T e39ab77df304447db021abaea59c6c7d P 11c27e06913849dd92d8124803c4a913: Bootstrap starting.
I20260807 10:54:26.465271 28176 log.cc:824] T e39ab77df304447db021abaea59c6c7d P 436e4f5a88964f58b970888835c9928c: Log is configured to *not* fsync() on all Append() calls
I20260807 10:54:26.467485 28177 tablet_bootstrap.cc:492] T e39ab77df304447db021abaea59c6c7d P a14d469484c94dd7baf344d9330b3fd3: Bootstrap starting.
I20260807 10:54:26.468782 28178 tablet_bootstrap.cc:654] T e39ab77df304447db021abaea59c6c7d P 11c27e06913849dd92d8124803c4a913: Neither blocks nor log segments found. Creating new log.
I20260807 10:54:26.468918 28176 tablet_bootstrap.cc:492] T e39ab77df304447db021abaea59c6c7d P 436e4f5a88964f58b970888835c9928c: No bootstrap required, opened a new log
I20260807 10:54:26.469197 28176 ts_tablet_manager.cc:1397] T e39ab77df304447db021abaea59c6c7d P 436e4f5a88964f58b970888835c9928c: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.002s
I20260807 10:54:26.470161 28178 log.cc:824] T e39ab77df304447db021abaea59c6c7d P 11c27e06913849dd92d8124803c4a913: Log is configured to *not* fsync() on all Append() calls
I20260807 10:54:26.471413 28177 tablet_bootstrap.cc:654] T e39ab77df304447db021abaea59c6c7d P a14d469484c94dd7baf344d9330b3fd3: Neither blocks nor log segments found. Creating new log.
I20260807 10:54:26.472817 28177 log.cc:824] T e39ab77df304447db021abaea59c6c7d P a14d469484c94dd7baf344d9330b3fd3: Log is configured to *not* fsync() on all Append() calls
I20260807 10:54:26.473743 28178 tablet_bootstrap.cc:492] T e39ab77df304447db021abaea59c6c7d P 11c27e06913849dd92d8124803c4a913: No bootstrap required, opened a new log
I20260807 10:54:26.474046 28178 ts_tablet_manager.cc:1397] T e39ab77df304447db021abaea59c6c7d P 11c27e06913849dd92d8124803c4a913: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260807 10:54:26.476051 28177 tablet_bootstrap.cc:492] T e39ab77df304447db021abaea59c6c7d P a14d469484c94dd7baf344d9330b3fd3: No bootstrap required, opened a new log
I20260807 10:54:26.476346 28177 ts_tablet_manager.cc:1397] T e39ab77df304447db021abaea59c6c7d P a14d469484c94dd7baf344d9330b3fd3: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260807 10:54:26.477797 28176 raft_consensus.cc:348] T e39ab77df304447db021abaea59c6c7d P 436e4f5a88964f58b970888835c9928c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "436e4f5a88964f58b970888835c9928c" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43253 } } peers { permanent_uuid: "a14d469484c94dd7baf344d9330b3fd3" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 36585 } } peers { permanent_uuid: "11c27e06913849dd92d8124803c4a913" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 46863 } }
I20260807 10:54:26.478338 28176 raft_consensus.cc:374] T e39ab77df304447db021abaea59c6c7d P 436e4f5a88964f58b970888835c9928c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:54:26.478444 28176 raft_consensus.cc:729] T e39ab77df304447db021abaea59c6c7d P 436e4f5a88964f58b970888835c9928c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 436e4f5a88964f58b970888835c9928c, State: Initialized, Role: FOLLOWER
I20260807 10:54:26.479216 28176 consensus_queue.cc:260] T e39ab77df304447db021abaea59c6c7d P 436e4f5a88964f58b970888835c9928c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "436e4f5a88964f58b970888835c9928c" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43253 } } peers { permanent_uuid: "a14d469484c94dd7baf344d9330b3fd3" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 36585 } } peers { permanent_uuid: "11c27e06913849dd92d8124803c4a913" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 46863 } }
I20260807 10:54:26.481496 27935 heartbeater.cc:503] Master 127.3.73.254:34761 was elected leader, sending a full tablet report...
I20260807 10:54:26.483495 28176 ts_tablet_manager.cc:1428] T e39ab77df304447db021abaea59c6c7d P 436e4f5a88964f58b970888835c9928c: Time spent starting tablet: real 0.014s user 0.011s sys 0.002s
I20260807 10:54:26.485185 28178 raft_consensus.cc:348] T e39ab77df304447db021abaea59c6c7d P 11c27e06913849dd92d8124803c4a913 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "436e4f5a88964f58b970888835c9928c" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43253 } } peers { permanent_uuid: "a14d469484c94dd7baf344d9330b3fd3" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 36585 } } peers { permanent_uuid: "11c27e06913849dd92d8124803c4a913" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 46863 } }
I20260807 10:54:26.485682 28178 raft_consensus.cc:374] T e39ab77df304447db021abaea59c6c7d P 11c27e06913849dd92d8124803c4a913 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:54:26.485785 28178 raft_consensus.cc:729] T e39ab77df304447db021abaea59c6c7d P 11c27e06913849dd92d8124803c4a913 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11c27e06913849dd92d8124803c4a913, State: Initialized, Role: FOLLOWER
I20260807 10:54:26.486450 28178 consensus_queue.cc:260] T e39ab77df304447db021abaea59c6c7d P 11c27e06913849dd92d8124803c4a913 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "436e4f5a88964f58b970888835c9928c" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43253 } } peers { permanent_uuid: "a14d469484c94dd7baf344d9330b3fd3" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 36585 } } peers { permanent_uuid: "11c27e06913849dd92d8124803c4a913" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 46863 } }
I20260807 10:54:26.487236 28177 raft_consensus.cc:348] T e39ab77df304447db021abaea59c6c7d P a14d469484c94dd7baf344d9330b3fd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "436e4f5a88964f58b970888835c9928c" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43253 } } peers { permanent_uuid: "a14d469484c94dd7baf344d9330b3fd3" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 36585 } } peers { permanent_uuid: "11c27e06913849dd92d8124803c4a913" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 46863 } }
I20260807 10:54:26.487666 28177 raft_consensus.cc:374] T e39ab77df304447db021abaea59c6c7d P a14d469484c94dd7baf344d9330b3fd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:54:26.487773 28177 raft_consensus.cc:729] T e39ab77df304447db021abaea59c6c7d P a14d469484c94dd7baf344d9330b3fd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a14d469484c94dd7baf344d9330b3fd3, State: Initialized, Role: FOLLOWER
I20260807 10:54:26.487874 28154 heartbeater.cc:503] Master 127.3.73.254:34761 was elected leader, sending a full tablet report...
I20260807 10:54:26.488338 28178 ts_tablet_manager.cc:1428] T e39ab77df304447db021abaea59c6c7d P 11c27e06913849dd92d8124803c4a913: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260807 10:54:26.488394 28177 consensus_queue.cc:260] T e39ab77df304447db021abaea59c6c7d P a14d469484c94dd7baf344d9330b3fd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "436e4f5a88964f58b970888835c9928c" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43253 } } peers { permanent_uuid: "a14d469484c94dd7baf344d9330b3fd3" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 36585 } } peers { permanent_uuid: "11c27e06913849dd92d8124803c4a913" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 46863 } }
I20260807 10:54:26.490000 28177 ts_tablet_manager.cc:1428] T e39ab77df304447db021abaea59c6c7d P a14d469484c94dd7baf344d9330b3fd3: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
W20260807 10:54:26.557291 27725 tablet.cc:2369] T e39ab77df304447db021abaea59c6c7d P a14d469484c94dd7baf344d9330b3fd3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 10:54:26.562970 27936 tablet.cc:2369] T e39ab77df304447db021abaea59c6c7d P 436e4f5a88964f58b970888835c9928c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 10:54:26.589718 28155 tablet.cc:2369] T e39ab77df304447db021abaea59c6c7d P 11c27e06913849dd92d8124803c4a913: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:54:26.878005 28186 raft_consensus.cc:482] T e39ab77df304447db021abaea59c6c7d P a14d469484c94dd7baf344d9330b3fd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 10:54:26.878360 28186 raft_consensus.cc:504] T e39ab77df304447db021abaea59c6c7d P a14d469484c94dd7baf344d9330b3fd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "436e4f5a88964f58b970888835c9928c" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43253 } } peers { permanent_uuid: "a14d469484c94dd7baf344d9330b3fd3" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 36585 } } peers { permanent_uuid: "11c27e06913849dd92d8124803c4a913" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 46863 } }
I20260807 10:54:26.880208 28186 leader_election.cc:288] T e39ab77df304447db021abaea59c6c7d P a14d469484c94dd7baf344d9330b3fd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 436e4f5a88964f58b970888835c9928c (127.3.73.194:43253), 11c27e06913849dd92d8124803c4a913 (127.3.73.195:46863)
I20260807 10:54:26.889628 28175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:26.880746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:57255 (local address 127.3.73.195:46863)
0807 10:54:26.880988 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 10:54:26.880996 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:54:26.881749 (+ 753us) server_negotiation.cc:407] Connection header received
0807 10:54:26.881948 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:54:26.881952 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:54:26.882016 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:54:26.882113 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:54:26.882883 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:26.884016 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:26.887101 (+ 3085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:26.888360 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:26.888534 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:26.888619 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 10:54:26.889219 (+ 600us) server_negotiation.cc:299] Negotiation successful
0807 10:54:26.889359 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 10:54:26.889628 28170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:26.880709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:46937 (local address 127.3.73.194:43253)
0807 10:54:26.880984 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0807 10:54:26.880991 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:54:26.881830 (+ 839us) server_negotiation.cc:407] Connection header received
0807 10:54:26.882045 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:54:26.882058 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:54:26.882121 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:54:26.882221 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:54:26.882943 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:26.884017 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:26.887101 (+ 3084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:26.888371 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:26.888532 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:26.888609 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 10:54:26.889219 (+ 610us) server_negotiation.cc:299] Negotiation successful
0807 10:54:26.889359 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 10:54:26.889884 28188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:26.880625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:43253 (local address 127.3.73.193:46937)
0807 10:54:26.881716 (+ 1091us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:26.881731 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 10:54:26.881902 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:54:26.882239 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:54:26.882248 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:54:26.882274 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:54:26.882713 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:54:26.882722 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:54:26.884162 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:54:26.884167 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:54:26.886918 (+ 2751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:54:26.886934 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:54:26.888663 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:54:26.888674 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:54:26.888996 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:26.889067 (+ 71us) client_negotiation.cc:769] Sending connection context
0807 10:54:26.889174 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 10:54:26.889325 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":979,"mutex_wait_us":159,"thread_start_us":777,"threads_started":1}
I20260807 10:54:26.890143 28187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:26.880581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:46863 (local address 127.3.73.193:57255)
0807 10:54:26.881574 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:26.881591 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 10:54:26.881779 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:54:26.882148 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:54:26.882153 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:54:26.882178 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:54:26.882706 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:54:26.882719 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:54:26.884163 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:54:26.884167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:54:26.886920 (+ 2753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:54:26.886934 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:54:26.888671 (+ 1737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:54:26.888676 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:54:26.888990 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:26.889068 (+ 78us) client_negotiation.cc:769] Sending connection context
0807 10:54:26.889174 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 10:54:26.889325 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":865,"mutex_wait_us":250,"spinlock_wait_cycles":36608,"thread_start_us":654,"threads_started":1}
I20260807 10:54:26.891100 28071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e39ab77df304447db021abaea59c6c7d" candidate_uuid: "a14d469484c94dd7baf344d9330b3fd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11c27e06913849dd92d8124803c4a913" is_pre_election: true
I20260807 10:54:26.891088 27861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e39ab77df304447db021abaea59c6c7d" candidate_uuid: "a14d469484c94dd7baf344d9330b3fd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "436e4f5a88964f58b970888835c9928c" is_pre_election: true
I20260807 10:54:26.891601 28071 raft_consensus.cc:2454] T e39ab77df304447db021abaea59c6c7d P 11c27e06913849dd92d8124803c4a913 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a14d469484c94dd7baf344d9330b3fd3 in term 0.
I20260807 10:54:26.891600 27861 raft_consensus.cc:2454] T e39ab77df304447db021abaea59c6c7d P 436e4f5a88964f58b970888835c9928c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a14d469484c94dd7baf344d9330b3fd3 in term 0.
I20260807 10:54:26.892347 27547 leader_election.cc:302] T e39ab77df304447db021abaea59c6c7d P a14d469484c94dd7baf344d9330b3fd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 11c27e06913849dd92d8124803c4a913, 436e4f5a88964f58b970888835c9928c, a14d469484c94dd7baf344d9330b3fd3; no voters:
I20260807 10:54:26.892882 28186 raft_consensus.cc:2781] T e39ab77df304447db021abaea59c6c7d P a14d469484c94dd7baf344d9330b3fd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 10:54:26.893067 28186 raft_consensus.cc:482] T e39ab77df304447db021abaea59c6c7d P a14d469484c94dd7baf344d9330b3fd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 10:54:26.893181 28186 raft_consensus.cc:3037] T e39ab77df304447db021abaea59c6c7d P a14d469484c94dd7baf344d9330b3fd3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:54:26.895473 28186 raft_consensus.cc:504] T e39ab77df304447db021abaea59c6c7d P a14d469484c94dd7baf344d9330b3fd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "436e4f5a88964f58b970888835c9928c" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43253 } } peers { permanent_uuid: "a14d469484c94dd7baf344d9330b3fd3" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 36585 } } peers { permanent_uuid: "11c27e06913849dd92d8124803c4a913" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 46863 } }
I20260807 10:54:26.896168 28186 leader_election.cc:288] T e39ab77df304447db021abaea59c6c7d P a14d469484c94dd7baf344d9330b3fd3 [CANDIDATE]: Term 1 election: Requested vote from peers 436e4f5a88964f58b970888835c9928c (127.3.73.194:43253), 11c27e06913849dd92d8124803c4a913 (127.3.73.195:46863)
I20260807 10:54:26.896684 27861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e39ab77df304447db021abaea59c6c7d" candidate_uuid: "a14d469484c94dd7baf344d9330b3fd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "436e4f5a88964f58b970888835c9928c"
I20260807 10:54:26.896893 28071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e39ab77df304447db021abaea59c6c7d" candidate_uuid: "a14d469484c94dd7baf344d9330b3fd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11c27e06913849dd92d8124803c4a913"
I20260807 10:54:26.896970 27861 raft_consensus.cc:3037] T e39ab77df304447db021abaea59c6c7d P 436e4f5a88964f58b970888835c9928c [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:54:26.897140 28071 raft_consensus.cc:3037] T e39ab77df304447db021abaea59c6c7d P 11c27e06913849dd92d8124803c4a913 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:54:26.899238 27861 raft_consensus.cc:2454] T e39ab77df304447db021abaea59c6c7d P 436e4f5a88964f58b970888835c9928c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a14d469484c94dd7baf344d9330b3fd3 in term 1.
I20260807 10:54:26.899416 28071 raft_consensus.cc:2454] T e39ab77df304447db021abaea59c6c7d P 11c27e06913849dd92d8124803c4a913 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a14d469484c94dd7baf344d9330b3fd3 in term 1.
I20260807 10:54:26.899673 27547 leader_election.cc:302] T e39ab77df304447db021abaea59c6c7d P a14d469484c94dd7baf344d9330b3fd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 436e4f5a88964f58b970888835c9928c, a14d469484c94dd7baf344d9330b3fd3; no voters:
I20260807 10:54:26.899981 28186 raft_consensus.cc:2781] T e39ab77df304447db021abaea59c6c7d P a14d469484c94dd7baf344d9330b3fd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:54:26.900413 28186 raft_consensus.cc:686] T e39ab77df304447db021abaea59c6c7d P a14d469484c94dd7baf344d9330b3fd3 [term 1 LEADER]: Becoming Leader. State: Replica: a14d469484c94dd7baf344d9330b3fd3, State: Running, Role: LEADER
I20260807 10:54:26.900992 28186 consensus_queue.cc:237] T e39ab77df304447db021abaea59c6c7d P a14d469484c94dd7baf344d9330b3fd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "436e4f5a88964f58b970888835c9928c" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43253 } } peers { permanent_uuid: "a14d469484c94dd7baf344d9330b3fd3" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 36585 } } peers { permanent_uuid: "11c27e06913849dd92d8124803c4a913" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 46863 } }
I20260807 10:54:26.906034 27447 catalog_manager.cc:5495] T e39ab77df304447db021abaea59c6c7d P a14d469484c94dd7baf344d9330b3fd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a14d469484c94dd7baf344d9330b3fd3 (127.3.73.193). New cstate: current_term: 1 leader_uuid: "a14d469484c94dd7baf344d9330b3fd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "436e4f5a88964f58b970888835c9928c" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a14d469484c94dd7baf344d9330b3fd3" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 36585 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11c27e06913849dd92d8124803c4a913" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 46863 } health_report { overall_health: UNKNOWN } } }
I20260807 10:54:27.006839 28196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:27.000655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42632 (local address 127.3.73.193:36585)
0807 10:54:27.001382 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0807 10:54:27.001394 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 10:54:27.001412 (+ 18us) server_negotiation.cc:407] Connection header received
0807 10:54:27.001505 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:54:27.001509 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:54:27.001595 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:54:27.001739 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:54:27.002116 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:27.002847 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:27.004352 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:27.004828 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:27.004940 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:27.005476 (+ 536us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:54:27.006151 (+ 675us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:54:27.006261 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 10:54:27.006418 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 10:54:27.006491 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":481,"threads_started":1}
I20260807 10:54:27.015776 27861 raft_consensus.cc:1261] T e39ab77df304447db021abaea59c6c7d P 436e4f5a88964f58b970888835c9928c [term 1 FOLLOWER]: Refusing update from remote peer a14d469484c94dd7baf344d9330b3fd3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:54:27.015807 28071 raft_consensus.cc:1261] T e39ab77df304447db021abaea59c6c7d P 11c27e06913849dd92d8124803c4a913 [term 1 FOLLOWER]: Refusing update from remote peer a14d469484c94dd7baf344d9330b3fd3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:54:27.016963 28189 consensus_queue.cc:1035] T e39ab77df304447db021abaea59c6c7d P a14d469484c94dd7baf344d9330b3fd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11c27e06913849dd92d8124803c4a913" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 46863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:54:27.017377 28186 consensus_queue.cc:1035] T e39ab77df304447db021abaea59c6c7d P a14d469484c94dd7baf344d9330b3fd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "436e4f5a88964f58b970888835c9928c" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:54:27.029183 28197 mvcc.cc:205] Tried to move back new op lower bound from 7315865874485456896 to 7315865874030981120. Current Snapshot: MvccSnapshot[applied={T|T < 7315865874485456896}]
I20260807 10:54:27.033819 28203 mvcc.cc:205] Tried to move back new op lower bound from 7315865874485456896 to 7315865874030981120. Current Snapshot: MvccSnapshot[applied={T|T < 7315865874485456896}]
I20260807 10:54:27.045746 27447 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:54:27.046046 27447 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:54:27.050220 27447 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb: Sending DeleteTablet for 3 replicas of tablet e39ab77df304447db021abaea59c6c7d
I20260807 10:54:27.051667 27831 tablet_service.cc:1514] Processing DeleteTablet for tablet e39ab77df304447db021abaea59c6c7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:54:27 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:34976
I20260807 10:54:27.051759 27626 tablet_service.cc:1514] Processing DeleteTablet for tablet e39ab77df304447db021abaea59c6c7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:54:27 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:42626
I20260807 10:54:27.052251 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 27499
I20260807 10:54:27.052876 28040 tablet_service.cc:1514] Processing DeleteTablet for tablet e39ab77df304447db021abaea59c6c7d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:54:27 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:60336
I20260807 10:54:27.062969 28215 tablet_replica.cc:330] T e39ab77df304447db021abaea59c6c7d P a14d469484c94dd7baf344d9330b3fd3: stopping tablet replica
I20260807 10:54:27.063529 28215 raft_consensus.cc:2229] T e39ab77df304447db021abaea59c6c7d P a14d469484c94dd7baf344d9330b3fd3 [term 1 LEADER]: Raft consensus shutting down.
I20260807 10:54:27.064306 28215 raft_consensus.cc:2258] T e39ab77df304447db021abaea59c6c7d P a14d469484c94dd7baf344d9330b3fd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:54:27.067098 28215 ts_tablet_manager.cc:1905] T e39ab77df304447db021abaea59c6c7d P a14d469484c94dd7baf344d9330b3fd3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:54:27.072237 28216 tablet_replica.cc:330] T e39ab77df304447db021abaea59c6c7d P 11c27e06913849dd92d8124803c4a913: stopping tablet replica
I20260807 10:54:27.072801 28216 raft_consensus.cc:2229] T e39ab77df304447db021abaea59c6c7d P 11c27e06913849dd92d8124803c4a913 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:54:27.073381 28216 raft_consensus.cc:2258] T e39ab77df304447db021abaea59c6c7d P 11c27e06913849dd92d8124803c4a913 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:54:27.075842 28215 ts_tablet_manager.cc:1918] T e39ab77df304447db021abaea59c6c7d P a14d469484c94dd7baf344d9330b3fd3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:54:27.076037 28215 log.cc:1196] T e39ab77df304447db021abaea59c6c7d P a14d469484c94dd7baf344d9330b3fd3: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/wal/wals/e39ab77df304447db021abaea59c6c7d
I20260807 10:54:27.076653 28215 ts_tablet_manager.cc:1939] T e39ab77df304447db021abaea59c6c7d P a14d469484c94dd7baf344d9330b3fd3: Deleting consensus metadata
I20260807 10:54:27.077311 28214 tablet_replica.cc:330] T e39ab77df304447db021abaea59c6c7d P 436e4f5a88964f58b970888835c9928c: stopping tablet replica
I20260807 10:54:27.077819 28214 raft_consensus.cc:2229] T e39ab77df304447db021abaea59c6c7d P 436e4f5a88964f58b970888835c9928c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:54:27.078349 28214 raft_consensus.cc:2258] T e39ab77df304447db021abaea59c6c7d P 436e4f5a88964f58b970888835c9928c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:54:27.078699 28216 ts_tablet_manager.cc:1905] T e39ab77df304447db021abaea59c6c7d P 11c27e06913849dd92d8124803c4a913: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:54:27.080111 28214 ts_tablet_manager.cc:1905] T e39ab77df304447db021abaea59c6c7d P 436e4f5a88964f58b970888835c9928c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:54:27.082947 27425 catalog_manager.cc:4841] TS a14d469484c94dd7baf344d9330b3fd3 (127.3.73.193:36585): tablet e39ab77df304447db021abaea59c6c7d (table test-table [id=0fc475f12f504114836c5b430e10888d]) successfully deleted
I20260807 10:54:27.086135 28216 ts_tablet_manager.cc:1918] T e39ab77df304447db021abaea59c6c7d P 11c27e06913849dd92d8124803c4a913: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:54:27.086310 28216 log.cc:1196] T e39ab77df304447db021abaea59c6c7d P 11c27e06913849dd92d8124803c4a913: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/wal/wals/e39ab77df304447db021abaea59c6c7d
I20260807 10:54:27.086941 28216 ts_tablet_manager.cc:1939] T e39ab77df304447db021abaea59c6c7d P 11c27e06913849dd92d8124803c4a913: Deleting consensus metadata
I20260807 10:54:27.088299 28214 ts_tablet_manager.cc:1918] T e39ab77df304447db021abaea59c6c7d P 436e4f5a88964f58b970888835c9928c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:54:27.088495 28214 log.cc:1196] T e39ab77df304447db021abaea59c6c7d P 436e4f5a88964f58b970888835c9928c: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/wal/wals/e39ab77df304447db021abaea59c6c7d
I20260807 10:54:27.088863 27425 catalog_manager.cc:4841] TS 11c27e06913849dd92d8124803c4a913 (127.3.73.195:46863): tablet e39ab77df304447db021abaea59c6c7d (table test-table [id=0fc475f12f504114836c5b430e10888d]) successfully deleted
I20260807 10:54:27.089083 28214 ts_tablet_manager.cc:1939] T e39ab77df304447db021abaea59c6c7d P 436e4f5a88964f58b970888835c9928c: Deleting consensus metadata
I20260807 10:54:27.090916 27426 catalog_manager.cc:4841] TS 436e4f5a88964f58b970888835c9928c (127.3.73.194:43253): tablet e39ab77df304447db021abaea59c6c7d (table test-table [id=0fc475f12f504114836c5b430e10888d]) successfully deleted
I20260807 10:54:27.091358 28196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:27.053836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42634 (local address 127.3.73.193:36585)
0807 10:54:27.054094 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0807 10:54:27.054104 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:54:27.070023 (+ 15919us) server_negotiation.cc:407] Connection header received
0807 10:54:27.070241 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:54:27.070251 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:54:27.070376 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:54:27.070535 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:54:27.074275 (+ 3740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:27.076061 (+ 1786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:27.078556 (+ 2495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:27.079257 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:27.079441 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:27.083449 (+ 4008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:54:27.083461 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:54:27.083475 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:54:27.083513 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:54:27.087355 (+ 3842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:54:27.088181 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:54:27.088195 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:54:27.088198 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:54:27.088352 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:54:27.088964 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:54:27.088970 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:54:27.088974 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:54:27.089457 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:54:27.089750 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0807 10:54:27.090380 (+ 630us) server_negotiation.cc:299] Negotiation successful
0807 10:54:27.090657 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 10:54:27.092130 27715 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:54:27.190683 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 27499
I20260807 10:54:27.221346 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 27728
I20260807 10:54:27.237696 28170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:27.222311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34980 (local address 127.3.73.194:43253)
0807 10:54:27.222546 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 10:54:27.222553 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:54:27.222577 (+ 24us) server_negotiation.cc:407] Connection header received
0807 10:54:27.222751 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:54:27.222755 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:54:27.222859 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:54:27.222986 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:54:27.224569 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:27.225464 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:27.227011 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:27.227489 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:27.227642 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:27.230764 (+ 3122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:54:27.230781 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:54:27.230798 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:54:27.230832 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:54:27.234458 (+ 3626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:54:27.235391 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:54:27.235402 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:54:27.235406 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:54:27.235558 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:54:27.235997 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:54:27.236002 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:54:27.236015 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:54:27.236420 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:54:27.236772 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0807 10:54:27.237088 (+ 316us) server_negotiation.cc:299] Negotiation successful
0807 10:54:27.237329 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 10:54:27.238536 27926 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:54:27.330246 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 27728
I20260807 10:54:27.367049 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 27943
I20260807 10:54:27.383698 28175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:27.367964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60340 (local address 127.3.73.195:46863)
0807 10:54:27.368152 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 10:54:27.368158 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:54:27.368180 (+ 22us) server_negotiation.cc:407] Connection header received
0807 10:54:27.368354 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:54:27.368360 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:54:27.368469 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:54:27.368587 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:54:27.370231 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:27.371413 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:27.373207 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:54:27.373875 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:54:27.374011 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:54:27.377175 (+ 3164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:54:27.377191 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:54:27.377208 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:54:27.377247 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:54:27.380607 (+ 3360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:54:27.381504 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:54:27.381513 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:54:27.381516 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:54:27.381665 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:54:27.382056 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:54:27.382060 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:54:27.382062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:54:27.382448 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:54:27.382695 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0807 10:54:27.383137 (+ 442us) server_negotiation.cc:299] Negotiation successful
0807 10:54:27.383360 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 10:54:27.384567 28143 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:54:27.481083 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 27943
I20260807 10:54:27.507236 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 27397
I20260807 10:54:27.508494 27491 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:54:27.626901 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 27397
I20260807 10:54:47.659400 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:34761
--webserver_interface=127.3.73.254
--webserver_port=38125
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:35801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:34761
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:54:48.050077 28222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:54:48.050427 28222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:54:48.050587 28222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:54:48.058436 28222 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 10:54:48.058557 28222 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 10:54:48.058624 28222 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 10:54:48.058703 28222 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:54:48.058796 28222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:54:48.058925 28222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:54:48.058993 28222 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:54:48.059082 28222 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:54:48.070005 28222 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:34761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:34761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=38125
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:54:48.071473 28222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:54:48.073053 28222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:54:48.088547 28230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:54:48.088840 28233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:54:48.088794 28231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:54:48.089633 28222 server_base.cc:1034] running on GCE node
I20260807 10:54:48.090659 28222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:54:48.092517 28222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:54:48.093845 28222 hybrid_clock.cc:648] HybridClock initialized: now 1786100088093768 us; error 66 us; skew 500 ppm
I20260807 10:54:48.094324 28222 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:54:48 dist-test-slave-x5fp krb5kdc[26544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100088, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:54:48.098650 28222 init.cc:377] Logged in from keytab as kudu/127.3.73.254@KRBTEST.COM (short username kudu)
I20260807 10:54:48.100582 28222 webserver.cc:460] Webserver started at http://127.3.73.254:38125/ using document root <none> and password file <none>
I20260807 10:54:48.101560 28222 fs_manager.cc:359] Metadata directory not provided
I20260807 10:54:48.101783 28222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:54:48.108582 28222 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.004s
I20260807 10:54:48.112072 28246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:54:48.113755 28222 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 10:54:48.113987 28222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "d3884469691e4d2fa1d13d24268c10fb"
format_stamp: "Formatted at 2026-08-07 10:54:01 on dist-test-slave-x5fp"
server_key: "df773170c018a442728ae50840739c8b"
server_key_iv: "ba29fa248fc99913c6384284143d3c36"
server_key_version: "encryptionkey@0"
I20260807 10:54:48.115116 28222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:54:48.136667 28222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:54:48.141458 28222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:54:48.141773 28222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:54:48.162153 28222 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:34761
I20260807 10:54:48.162216 28333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:34761 every 8 connection(s)
I20260807 10:54:48.163790 28222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:54:48.170272 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 28222
I20260807 10:54:48.171487 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.193:36585
--local_ip_for_outbound_sockets=127.3.73.193
--tserver_master_addrs=127.3.73.254:34761
--webserver_port=45709
--webserver_interface=127.3.73.193
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:35801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 10:54:48.173154 28335 sys_catalog.cc:263] Verifying existing consensus state
I20260807 10:54:48.178829 28335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb: Bootstrap starting.
I20260807 10:54:48.202934 28335 log.cc:824] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb: Log is configured to *not* fsync() on all Append() calls
I20260807 10:54:48.220608 28335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 10:54:48.221273 28335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb: Bootstrap complete.
I20260807 10:54:48.231717 28335 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3884469691e4d2fa1d13d24268c10fb" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 34761 } }
I20260807 10:54:48.232722 28335 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3884469691e4d2fa1d13d24268c10fb, State: Initialized, Role: FOLLOWER
I20260807 10:54:48.233393 28335 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3884469691e4d2fa1d13d24268c10fb" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 34761 } }
I20260807 10:54:48.233645 28335 raft_consensus.cc:388] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:54:48.233765 28335 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:54:48.233973 28335 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [term 2 FOLLOWER]: Advancing to term 3
I20260807 10:54:48.238165 28335 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3884469691e4d2fa1d13d24268c10fb" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 34761 } }
I20260807 10:54:48.238729 28335 leader_election.cc:302] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d3884469691e4d2fa1d13d24268c10fb; no voters:
I20260807 10:54:48.239467 28335 leader_election.cc:288] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 10:54:48.239617 28341 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [term 3 FOLLOWER]: Leader election won for term 3
I20260807 10:54:48.240986 28341 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [term 3 LEADER]: Becoming Leader. State: Replica: d3884469691e4d2fa1d13d24268c10fb, State: Running, Role: LEADER
I20260807 10:54:48.241606 28341 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3884469691e4d2fa1d13d24268c10fb" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 34761 } }
I20260807 10:54:48.242010 28335 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:54:48.244493 28343 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "d3884469691e4d2fa1d13d24268c10fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3884469691e4d2fa1d13d24268c10fb" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 34761 } } }
I20260807 10:54:48.244910 28343 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [sys.catalog]: This master's current role is: LEADER
I20260807 10:54:48.245261 28344 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3884469691e4d2fa1d13d24268c10fb. Latest consensus state: current_term: 3 leader_uuid: "d3884469691e4d2fa1d13d24268c10fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d3884469691e4d2fa1d13d24268c10fb" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 34761 } } }
I20260807 10:54:48.245754 28344 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb [sys.catalog]: This master's current role is: LEADER
I20260807 10:54:48.257735 28349 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:54:48.270394 28349 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:54:48.272620 28349 catalog_manager.cc:1259] Loaded cluster ID: a0a7f60e516044948ef21b7d4609ba74
I20260807 10:54:48.272821 28349 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:54:48.278679 28349 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:54:48.281440 28349 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb: Loaded TSK: 0
I20260807 10:54:48.281662 28349 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d3884469691e4d2fa1d13d24268c10fb: Loaded TSK: 1
F20260807 10:54:48.283303 28349 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f6ca925b9ad google::LogMessage::Fail() at ??:0
@ 0x7f6ca925e88a google::LogMessage::SendToLog() at ??:0
@ 0x7f6ca925b4de google::LogMessage::Flush() at ??:0
@ 0x7f6ca925ca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f6cacfbf7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f6cacfae73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f6cbec8e1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f6cbec8d783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f6cbed192a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f6cbed190c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f6cbed635e9 std::function<>::operator()() at ??:0
@ 0x7f6cbec91cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f6cbec8fc36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f6cbed17610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f6cbed17439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f6cabea7345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f6cabeaa91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f6cabeaa799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f6cabe85eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f6caf15f6db start_thread at ??:0
@ 0x7f6ca80f561f clone at ??:0
W20260807 10:54:48.638160 28338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:54:48.638525 28338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:54:48.638701 28338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:54:48.647178 28338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:54:48.647403 28338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:54:48.647604 28338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.193
I20260807 10:54:48.659085 28338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.193:36585
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.73.193
--webserver_port=45709
--tserver_master_addrs=127.3.73.254:34761
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.193
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:54:48.660588 28338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:54:48.662161 28338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:54:48.677032 28405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:54:48.677129 28400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:54:48.677170 28401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:54:48.678404 28338 server_base.cc:1034] running on GCE node
I20260807 10:54:48.679212 28338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:54:48.680429 28338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:54:48.681684 28338 hybrid_clock.cc:648] HybridClock initialized: now 1786100088681597 us; error 75 us; skew 500 ppm
I20260807 10:54:48.682029 28338 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:54:48 dist-test-slave-x5fp krb5kdc[26544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100088, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:54:48.685660 28338 init.cc:377] Logged in from keytab as kudu/127.3.73.193@KRBTEST.COM (short username kudu)
I20260807 10:54:48.687594 28338 webserver.cc:460] Webserver started at http://127.3.73.193:45709/ using document root <none> and password file <none>
I20260807 10:54:48.688413 28338 fs_manager.cc:359] Metadata directory not provided
I20260807 10:54:48.688587 28338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:54:48.694754 28338 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 10:54:48.697888 28415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:54:48.699352 28338 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 10:54:48.699558 28338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/wal
uuid: "a14d469484c94dd7baf344d9330b3fd3"
format_stamp: "Formatted at 2026-08-07 10:54:02 on dist-test-slave-x5fp"
server_key: "ce4aec35ac94456d43fd1e1ee2679a92"
server_key_iv: "6f96675806152327a853aeb9e18c19dc"
server_key_version: "encryptionkey@0"
I20260807 10:54:48.700611 28338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:54:48.725066 28338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:54:48.729368 28338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:54:48.729694 28338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:54:48.730829 28338 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:54:48.732479 28338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:54:48.732609 28338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:54:48.732766 28338 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:54:48.732859 28338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260807 10:54:48.770093 28338 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.193:36585
I20260807 10:54:48.770186 28588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.193:36585 every 8 connection(s)
I20260807 10:54:48.771680 28338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
I20260807 10:54:48.776926 28593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:48.774894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.193:47405)
0807 10:54:48.775651 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:48.775897 (+ 246us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":291,"threads_started":1}
W20260807 10:54:48.777623 28591 heartbeater.cc:644] Failed to heartbeat to 127.3.73.254:34761 (0 consecutive failures): Network error: Failed to ping master at 127.3.73.254:34761: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
I20260807 10:54:48.778491 28593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:48.778159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.193:53583)
0807 10:54:48.778302 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:48.778379 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 10:54:48.779284 28593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:48.778970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.193:58047)
0807 10:54:48.779106 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:48.779179 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 10:54:48.779511 28591 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 10:54:48.781073 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 28338
I20260807 10:54:48.781744 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.194:43253
--local_ip_for_outbound_sockets=127.3.73.194
--tserver_master_addrs=127.3.73.254:34761
--webserver_port=38319
--webserver_interface=127.3.73.194
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:35801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:54:49.168574 28594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:54:49.168969 28594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:54:49.169080 28594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:54:49.177666 28594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:54:49.177822 28594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:54:49.177955 28594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.194
I20260807 10:54:49.189785 28594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.194:43253
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.73.194
--webserver_port=38319
--tserver_master_addrs=127.3.73.254:34761
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.194
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:54:49.191232 28594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:54:49.192757 28594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:54:49.207976 28601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:54:49.208081 28602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:54:49.208009 28606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:54:49.209095 28594 server_base.cc:1034] running on GCE node
I20260807 10:54:49.209812 28594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:54:49.210970 28594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:54:49.212230 28594 hybrid_clock.cc:648] HybridClock initialized: now 1786100089212156 us; error 66 us; skew 500 ppm
I20260807 10:54:49.212605 28594 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:54:49 dist-test-slave-x5fp krb5kdc[26544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100089, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:54:49.216168 28594 init.cc:377] Logged in from keytab as kudu/127.3.73.194@KRBTEST.COM (short username kudu)
I20260807 10:54:49.217919 28594 webserver.cc:460] Webserver started at http://127.3.73.194:38319/ using document root <none> and password file <none>
I20260807 10:54:49.218657 28594 fs_manager.cc:359] Metadata directory not provided
I20260807 10:54:49.218771 28594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:54:49.225066 28594 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 10:54:49.228473 28620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:54:49.230144 28594 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 10:54:49.230397 28594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/wal
uuid: "436e4f5a88964f58b970888835c9928c"
format_stamp: "Formatted at 2026-08-07 10:54:02 on dist-test-slave-x5fp"
server_key: "c141ad2f0662218af90dde747a70f1ec"
server_key_iv: "8c7bf8e5a7cd98dbef9db3d5f4a08264"
server_key_version: "encryptionkey@0"
I20260807 10:54:49.231501 28594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:54:49.263901 28594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:54:49.269276 28594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:54:49.269601 28594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:54:49.270843 28594 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:54:49.272511 28594 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:54:49.272608 28594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:54:49.272789 28594 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:54:49.272884 28594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:54:49.309919 28594 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.194:43253
I20260807 10:54:49.310046 28808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.194:43253 every 8 connection(s)
I20260807 10:54:49.311497 28594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
I20260807 10:54:49.317279 28815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:49.315070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.194:49065)
0807 10:54:49.315945 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:49.316189 (+ 244us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":330,"threads_started":1}
W20260807 10:54:49.318199 28812 heartbeater.cc:644] Failed to heartbeat to 127.3.73.254:34761 (0 consecutive failures): Network error: Failed to ping master at 127.3.73.254:34761: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
I20260807 10:54:49.318969 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 28594
I20260807 10:54:49.319540 28815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:49.318986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.194:33975)
0807 10:54:49.319312 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:49.319404 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176}
I20260807 10:54:49.319669 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.195:46863
--local_ip_for_outbound_sockets=127.3.73.195
--tserver_master_addrs=127.3.73.254:34761
--webserver_port=34619
--webserver_interface=127.3.73.195
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:35801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 10:54:49.320633 28815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:49.320198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.194:43337)
0807 10:54:49.320393 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:49.320496 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260807 10:54:49.321007 28812 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 10:54:49.700488 28818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:54:49.700978 28818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:54:49.701195 28818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:54:49.715158 28818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:54:49.715346 28818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:54:49.715529 28818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.195
I20260807 10:54:49.726732 28818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.195:46863
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.73.195
--webserver_port=34619
--tserver_master_addrs=127.3.73.254:34761
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.195
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:54:49.728878 28818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:54:49.730435 28818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:54:49.745159 28824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:54:49.745200 28826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:54:49.745380 28829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:54:49.746567 28818 server_base.cc:1034] running on GCE node
I20260807 10:54:49.747336 28818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:54:49.748502 28818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:54:49.749770 28818 hybrid_clock.cc:648] HybridClock initialized: now 1786100089749720 us; error 75 us; skew 500 ppm
I20260807 10:54:49.750151 28818 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:54:49 dist-test-slave-x5fp krb5kdc[26544](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100089, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:54:49.753875 28818 init.cc:377] Logged in from keytab as kudu/127.3.73.195@KRBTEST.COM (short username kudu)
I20260807 10:54:49.755712 28818 webserver.cc:460] Webserver started at http://127.3.73.195:34619/ using document root <none> and password file <none>
I20260807 10:54:49.756474 28818 fs_manager.cc:359] Metadata directory not provided
I20260807 10:54:49.756593 28818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:54:49.762616 28818 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 10:54:49.766003 28841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:54:49.767609 28818 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 10:54:49.767822 28818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/wal
uuid: "11c27e06913849dd92d8124803c4a913"
format_stamp: "Formatted at 2026-08-07 10:54:03 on dist-test-slave-x5fp"
server_key: "bc9543176defdd68d1cca3a24cc19511"
server_key_iv: "5ddaef1ce08b1eacd0bfe97cd67faac4"
server_key_version: "encryptionkey@0"
I20260807 10:54:49.768996 28818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:54:49.781232 28850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:49.780215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.193:40093)
0807 10:54:49.780775 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:49.780907 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":247,"threads_started":1}
I20260807 10:54:49.784027 28818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:54:49.788256 28818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:54:49.788518 28818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:54:49.789664 28818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:54:49.791155 28818 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:54:49.791261 28818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:54:49.791339 28818 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:54:49.791395 28818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:54:49.827419 28818 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.195:46863
I20260807 10:54:49.827543 29016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.195:46863 every 8 connection(s)
I20260807 10:54:49.829087 28818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
I20260807 10:54:49.834069 29021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:49.832052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.195:48029)
0807 10:54:49.832859 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:49.833094 (+ 235us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":282,"threads_started":1}
W20260807 10:54:49.834785 29017 heartbeater.cc:644] Failed to heartbeat to 127.3.73.254:34761 (0 consecutive failures): Network error: Failed to ping master at 127.3.73.254:34761: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
I20260807 10:54:49.835778 29021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:49.835407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.195:37817)
0807 10:54:49.835595 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:49.835666 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260807 10:54:49.836776 29021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:49.836362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.195:43429)
0807 10:54:49.836507 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:49.836614 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 10:54:49.837033 29017 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 10:54:49.838773 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 28818
I20260807 10:54:50.322566 29023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:50.321666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.194:46269)
0807 10:54:50.322201 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:50.322293 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":282,"threads_started":1}
I20260807 10:54:50.783258 29024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:50.782319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.193:45565)
0807 10:54:50.782976 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:50.783066 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":303,"threads_started":1}
I20260807 10:54:50.838639 29027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:50.837670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.195:59355)
0807 10:54:50.838244 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:50.838374 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":270,"threads_started":1}
I20260807 10:54:51.324447 29029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:51.323654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.194:47285)
0807 10:54:51.324171 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:51.324259 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":260,"threads_started":1}
I20260807 10:54:51.785269 29030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:51.784312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.193:39853)
0807 10:54:51.784900 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:51.784999 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":338,"threads_started":1}
I20260807 10:54:51.840565 29031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:51.839754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.195:41871)
0807 10:54:51.840289 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:51.840386 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":240,"threads_started":1}
I20260807 10:54:52.326234 29032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:52.325451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.194:57757)
0807 10:54:52.325963 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:52.326049 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":275,"threads_started":1}
I20260807 10:54:52.787081 29033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:52.786283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.193:34397)
0807 10:54:52.786810 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:52.786896 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":298,"threads_started":1}
I20260807 10:54:52.842231 29034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:52.841452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.195:35149)
0807 10:54:52.841955 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:52.842048 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":269,"threads_started":1}
I20260807 10:54:53.328320 29035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:53.327502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.194:54927)
0807 10:54:53.328034 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:53.328131 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":271,"threads_started":1}
I20260807 10:54:53.789258 29036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:53.788184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.193:45285)
0807 10:54:53.788953 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:53.789059 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":273,"threads_started":1}
I20260807 10:54:53.844158 29037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:53.843323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.195:55527)
0807 10:54:53.843841 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:53.843944 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":258,"threads_started":1}
I20260807 10:54:54.330523 29039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:54.329578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.194:46285)
0807 10:54:54.330160 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:54.330262 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":318,"threads_started":1}
I20260807 10:54:54.791486 29040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:54.790467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.193:49791)
0807 10:54:54.791161 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:54.791290 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":329,"threads_started":1}
I20260807 10:54:54.846259 29042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:54.845399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.195:48463)
0807 10:54:54.845974 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:54.846070 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":319,"threads_started":1}
I20260807 10:54:55.332628 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:55.331685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.194:49767)
0807 10:54:55.332321 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:55.332446 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":348,"threads_started":1}
I20260807 10:54:55.793725 29044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:55.792683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.193:41213)
0807 10:54:55.793329 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:55.793463 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":265,"threads_started":1}
I20260807 10:54:55.848059 29045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:55.847215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.195:52385)
0807 10:54:55.847716 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:55.847870 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":264,"threads_started":1}
I20260807 10:54:56.335374 29046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:56.333954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.194:40981)
0807 10:54:56.334926 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:56.335064 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":782,"thread_start_us":590,"threads_started":1}
I20260807 10:54:56.795933 29048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:56.794972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.193:42845)
0807 10:54:56.795614 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:56.795741 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":376,"threads_started":1}
I20260807 10:54:56.850175 29049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:56.849301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.195:33705)
0807 10:54:56.849850 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:56.849943 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":244,"threads_started":1}
I20260807 10:54:57.337567 29050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:57.336559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.194:45887)
0807 10:54:57.337201 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:57.337302 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":355,"threads_started":1}
I20260807 10:54:57.798095 29051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:57.797200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.193:33645)
0807 10:54:57.797757 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:57.797849 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":287,"threads_started":1}
I20260807 10:54:57.852330 29052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:57.851412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.195:38775)
0807 10:54:57.852041 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:57.852134 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":381,"threads_started":1}
I20260807 10:54:58.339599 29055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:58.338641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.194:36409)
0807 10:54:58.339246 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:58.339386 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":280,"threads_started":1}
I20260807 10:54:58.800215 29058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:58.799373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.193:44027)
0807 10:54:58.799940 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:58.800035 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":280,"threads_started":1}
I20260807 10:54:58.854185 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:58.853403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.195:45681)
0807 10:54:58.853909 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:58.853999 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":242,"threads_started":1}
I20260807 10:54:59.341627 29062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:59.340801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.194:50161)
0807 10:54:59.341350 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:59.341448 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":287,"threads_started":1}
I20260807 10:54:59.802448 29064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:59.801511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.193:46207)
0807 10:54:59.802117 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:59.802231 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":324,"threads_started":1}
I20260807 10:54:59.855990 29065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:54:59.855231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.195:46919)
0807 10:54:59.855703 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0807 10:54:59.855787 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":258,"threads_started":1}
I20260807 10:55:00.343713 29067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:00.342804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.194:48687)
0807 10:55:00.343397 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:00.343512 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":306,"threads_started":1}
I20260807 10:55:00.804550 29068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:00.803615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.193:40299)
0807 10:55:00.804201 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:00.804306 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":285,"threads_started":1}
I20260807 10:55:00.858114 29071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:00.857137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.195:49839)
0807 10:55:00.857752 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:00.857878 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":302,"threads_started":1}
I20260807 10:55:01.345994 29073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:01.345071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.194:40815)
0807 10:55:01.345701 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:01.345801 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":325,"threads_started":1}
I20260807 10:55:01.806761 29074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:01.805809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.193:54647)
0807 10:55:01.806442 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:01.806555 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":348,"threads_started":1}
I20260807 10:55:01.860206 29075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:01.859385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.195:53547)
0807 10:55:01.859935 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:01.860030 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":268,"threads_started":1}
I20260807 10:55:02.348361 29077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:02.347348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.194:44539)
0807 10:55:02.348010 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:02.348130 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":298,"threads_started":1}
I20260807 10:55:02.808857 29078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:02.807884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.193:51111)
0807 10:55:02.808446 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:02.808540 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":326,"threads_started":1}
I20260807 10:55:02.862337 29079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:02.861405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.195:36355)
0807 10:55:02.862007 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:02.862130 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":265,"threads_started":1}
I20260807 10:55:03.350589 29082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:03.349677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.194:45757)
0807 10:55:03.350269 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:03.350374 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":320,"threads_started":1}
I20260807 10:55:03.811314 29083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:03.810322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.193:46107)
0807 10:55:03.810904 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:03.811075 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":303,"threads_started":1}
I20260807 10:55:03.864323 29084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:03.863471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.195:41933)
0807 10:55:03.864038 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:03.864127 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":314,"threads_started":1}
I20260807 10:55:04.353129 29085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:04.352094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.194:53533)
0807 10:55:04.352804 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:04.352921 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":368,"threads_started":1}
I20260807 10:55:04.813408 29087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:04.812447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.193:40517)
0807 10:55:04.813088 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:04.813190 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":377,"threads_started":1}
I20260807 10:55:04.845233 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 28338
I20260807 10:55:04.866611 29089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:04.865618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34761 (local address 127.3.73.195:43963)
0807 10:55:04.866287 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:04.866399 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.254:34761: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":356,"threads_started":1}
I20260807 10:55:04.873132 29088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:04.846894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53068 (local address 127.3.73.193:36585)
0807 10:55:04.847671 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:04.847686 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:04.847734 (+ 48us) server_negotiation.cc:407] Connection header received
0807 10:55:04.849493 (+ 1759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:04.849503 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:04.849951 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:04.850283 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:04.851981 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:04.853462 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:04.855308 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:04.856021 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:04.856177 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:04.865261 (+ 9084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:04.865272 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:04.865291 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:55:04.865359 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:04.868945 (+ 3586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:04.869836 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:04.869845 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:04.869855 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:04.870049 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:04.870509 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:04.870515 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:04.870518 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:04.871127 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:04.871485 (+ 358us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:04.871999 (+ 514us) server_negotiation.cc:299] Negotiation successful
0807 10:55:04.872706 (+ 707us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":420,"threads_started":1}
I20260807 10:55:04.874826 28582 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:55:04.965823 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 28338
I20260807 10:55:04.997834 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 28594
I20260807 10:55:05.019174 29095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:04.998987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60024 (local address 127.3.73.194:43253)
0807 10:55:04.999561 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:04.999576 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:04.999616 (+ 40us) server_negotiation.cc:407] Connection header received
0807 10:55:05.001462 (+ 1846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:05.001475 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:05.001975 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:05.002391 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:05.004146 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:05.005788 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:05.007209 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:05.007982 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:05.008219 (+ 237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:05.011397 (+ 3178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:05.011407 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:05.011547 (+ 140us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:55:05.011619 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:05.015205 (+ 3586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:05.016133 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:05.016147 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:05.016155 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:05.016383 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:05.016873 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:05.016879 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:05.016882 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:05.017499 (+ 617us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:05.017852 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:05.018381 (+ 529us) server_negotiation.cc:299] Negotiation successful
0807 10:55:05.018861 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":281,"threads_started":1}
I20260807 10:55:05.020713 28798 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:55:05.109668 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 28594
I20260807 10:55:05.136143 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 28818
I20260807 10:55:05.157753 29100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:05.137515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60262 (local address 127.3.73.195:46863)
0807 10:55:05.138206 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:05.138222 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:05.138268 (+ 46us) server_negotiation.cc:407] Connection header received
0807 10:55:05.139981 (+ 1713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:05.140035 (+ 54us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:05.140638 (+ 603us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:05.141089 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:05.142727 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:05.144258 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:05.145955 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:05.146737 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:05.146957 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:05.150425 (+ 3468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:05.150434 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:05.150454 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:55:05.150503 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:05.153942 (+ 3439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:05.154849 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:05.154859 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:05.154866 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:05.155073 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:05.155491 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:05.155495 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:05.155497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:05.156102 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:05.156394 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:05.156912 (+ 518us) server_negotiation.cc:299] Negotiation successful
0807 10:55:05.157421 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":362,"threads_started":1}
I20260807 10:55:05.159301 29012 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:55:05.245868 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 28818
2026-08-07T10:55:05Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (66224 ms)
[----------] 30 tests from SecurityITest (337385 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:55:05 dist-test-slave-x5fp krb5kdc[29105](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:55:05 dist-test-slave-x5fp krb5kdc[29105](info): set up 2 sockets
Aug 07 10:55:05 dist-test-slave-x5fp krb5kdc[29105](info): commencing operation
krb5kdc: starting...
W20260807 10:55:06.653285 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.356s user 0.003s sys 0.057s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:55:06 dist-test-slave-x5fp krb5kdc[29105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:55:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:55:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.73.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
I20260807 10:55:07.038089 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:34889
--webserver_interface=127.3.73.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:37563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:34889
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:55:07.464439 29126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:55:07.464932 29126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:55:07.465122 29126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:55:07.474635 29126 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:55:07.474884 29126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:55:07.475024 29126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:55:07.475142 29126 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:55:07.475229 29126 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:55:07.488428 29126 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:37563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:34889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:34889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:55:07.490382 29126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:55:07.492038 29126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:55:07.508663 29131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:07.508741 29132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:07.508755 29136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:07.510238 29126 server_base.cc:1034] running on GCE node
I20260807 10:55:07.511293 29126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:55:07.513192 29126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:55:07.514494 29126 hybrid_clock.cc:648] HybridClock initialized: now 1786100107514460 us; error 53 us; skew 500 ppm
I20260807 10:55:07.514959 29126 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:55:07 dist-test-slave-x5fp krb5kdc[29105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100107, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:55:07.518870 29126 init.cc:377] Logged in from keytab as kudu/127.3.73.254@KRBTEST.COM (short username kudu)
I20260807 10:55:07.521420 29126 webserver.cc:460] Webserver started at http://127.3.73.254:42743/ using document root <none> and password file <none>
I20260807 10:55:07.522455 29126 fs_manager.cc:359] Metadata directory not provided
I20260807 10:55:07.522666 29126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:55:07.523068 29126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:55:07.527268 29126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "7ba6a47e245f47ad8e7fd7315430f9a6"
format_stamp: "Formatted at 2026-08-07 10:55:07 on dist-test-slave-x5fp"
server_key: "d93883df428e9080d1d1fec3821fe9a9"
server_key_iv: "6d099a0ed293de1f96ab8c27bd781c4e"
server_key_version: "encryptionkey@0"
I20260807 10:55:07.528368 29126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "7ba6a47e245f47ad8e7fd7315430f9a6"
format_stamp: "Formatted at 2026-08-07 10:55:07 on dist-test-slave-x5fp"
server_key: "d93883df428e9080d1d1fec3821fe9a9"
server_key_iv: "6d099a0ed293de1f96ab8c27bd781c4e"
server_key_version: "encryptionkey@0"
I20260807 10:55:07.535601 29126 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 10:55:07.541633 29155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:07.544552 29126 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.000s
I20260807 10:55:07.544955 29126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "7ba6a47e245f47ad8e7fd7315430f9a6"
format_stamp: "Formatted at 2026-08-07 10:55:07 on dist-test-slave-x5fp"
server_key: "d93883df428e9080d1d1fec3821fe9a9"
server_key_iv: "6d099a0ed293de1f96ab8c27bd781c4e"
server_key_version: "encryptionkey@0"
I20260807 10:55:07.545269 29126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:55:07.565452 29126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:55:07.571841 29126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:55:07.572257 29126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:55:07.596637 29126 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:34889
I20260807 10:55:07.596671 29238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:34889 every 8 connection(s)
I20260807 10:55:07.598373 29126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:55:07.599979 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 29126
I20260807 10:55:07.600428 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:55:07.601372 3367 external_mini_cluster.cc:1428] Setting key f312a9f568a4baaafbfbd4e9a835c383
I20260807 10:55:07.604323 29239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:55:07.618791 29239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ba6a47e245f47ad8e7fd7315430f9a6: Bootstrap starting.
Aug 07 10:55:07 dist-test-slave-x5fp krb5kdc[29105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:55:07.622661 29239 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ba6a47e245f47ad8e7fd7315430f9a6: Neither blocks nor log segments found. Creating new log.
I20260807 10:55:07.623986 29239 log.cc:824] T 00000000000000000000000000000000 P 7ba6a47e245f47ad8e7fd7315430f9a6: Log is configured to *not* fsync() on all Append() calls
I20260807 10:55:07.627401 29239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ba6a47e245f47ad8e7fd7315430f9a6: No bootstrap required, opened a new log
I20260807 10:55:07.630635 29242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:07.605394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56306 (local address 127.3.73.254:34889)
0807 10:55:07.607259 (+ 1865us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:07.607276 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:07.607335 (+ 59us) server_negotiation.cc:407] Connection header received
0807 10:55:07.609979 (+ 2644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:07.610007 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:07.610497 (+ 490us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:07.610987 (+ 490us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:07.612825 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:07.614199 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:07.615681 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:07.616519 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:07.616761 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:07.621895 (+ 5134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:07.621905 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:07.621923 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:55:07.621975 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:07.625843 (+ 3868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:07.626781 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:07.626788 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:07.626795 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:07.626982 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:07.627434 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:07.627439 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:07.627443 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:07.628115 (+ 672us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:07.628481 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:07.628990 (+ 509us) server_negotiation.cc:299] Negotiation successful
0807 10:55:07.629497 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1550,"thread_start_us":1261,"threads_started":1}
I20260807 10:55:07.636116 29239 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ba6a47e245f47ad8e7fd7315430f9a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ba6a47e245f47ad8e7fd7315430f9a6" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 34889 } }
I20260807 10:55:07.636569 29239 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ba6a47e245f47ad8e7fd7315430f9a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:55:07.636677 29239 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ba6a47e245f47ad8e7fd7315430f9a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ba6a47e245f47ad8e7fd7315430f9a6, State: Initialized, Role: FOLLOWER
I20260807 10:55:07.637365 29239 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ba6a47e245f47ad8e7fd7315430f9a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ba6a47e245f47ad8e7fd7315430f9a6" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 34889 } }
I20260807 10:55:07.637610 29239 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ba6a47e245f47ad8e7fd7315430f9a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:55:07.637784 29239 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ba6a47e245f47ad8e7fd7315430f9a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:55:07.637979 29239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ba6a47e245f47ad8e7fd7315430f9a6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:55:07.640519 29239 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ba6a47e245f47ad8e7fd7315430f9a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ba6a47e245f47ad8e7fd7315430f9a6" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 34889 } }
I20260807 10:55:07.641201 29239 leader_election.cc:302] T 00000000000000000000000000000000 P 7ba6a47e245f47ad8e7fd7315430f9a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ba6a47e245f47ad8e7fd7315430f9a6; no voters:
I20260807 10:55:07.642031 29239 leader_election.cc:288] T 00000000000000000000000000000000 P 7ba6a47e245f47ad8e7fd7315430f9a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:55:07.642195 29246 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ba6a47e245f47ad8e7fd7315430f9a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:55:07.643031 29246 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ba6a47e245f47ad8e7fd7315430f9a6 [term 1 LEADER]: Becoming Leader. State: Replica: 7ba6a47e245f47ad8e7fd7315430f9a6, State: Running, Role: LEADER
I20260807 10:55:07.643656 29246 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ba6a47e245f47ad8e7fd7315430f9a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ba6a47e245f47ad8e7fd7315430f9a6" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 34889 } }
I20260807 10:55:07.644459 29239 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ba6a47e245f47ad8e7fd7315430f9a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:55:07.648398 29248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ba6a47e245f47ad8e7fd7315430f9a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ba6a47e245f47ad8e7fd7315430f9a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ba6a47e245f47ad8e7fd7315430f9a6" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 34889 } } }
I20260807 10:55:07.648864 29248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ba6a47e245f47ad8e7fd7315430f9a6 [sys.catalog]: This master's current role is: LEADER
I20260807 10:55:07.649775 29259 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:55:07.649669 29250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ba6a47e245f47ad8e7fd7315430f9a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ba6a47e245f47ad8e7fd7315430f9a6. Latest consensus state: current_term: 1 leader_uuid: "7ba6a47e245f47ad8e7fd7315430f9a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ba6a47e245f47ad8e7fd7315430f9a6" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 34889 } } }
I20260807 10:55:07.650094 29250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ba6a47e245f47ad8e7fd7315430f9a6 [sys.catalog]: This master's current role is: LEADER
I20260807 10:55:07.656092 29259 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:55:07.666774 29259 catalog_manager.cc:1347] Generated new cluster ID: b87027a192c940fcb76a4ee0fa40a066
I20260807 10:55:07.666987 29259 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:55:07.684583 29259 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:55:07.686229 29259 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:55:07.698326 29259 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ba6a47e245f47ad8e7fd7315430f9a6: Generated new TSK 0
I20260807 10:55:07.699627 29259 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.73.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab.
Entry for principal kudu/127.3.73.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab.
I20260807 10:55:07.802158 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.193:0
--local_ip_for_outbound_sockets=127.3.73.193
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:34889
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:37563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:55:08.198490 29271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:55:08.198880 29271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:55:08.198987 29271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:55:08.206851 29271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:55:08.206995 29271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:55:08.207115 29271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.193
I20260807 10:55:08.218281 29271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:37563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:34889
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.193
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:55:08.219714 29271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:55:08.221196 29271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:55:08.237114 29279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:08.237246 29280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:08.237152 29285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:08.238683 29271 server_base.cc:1034] running on GCE node
I20260807 10:55:08.239511 29271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:55:08.240686 29271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:55:08.242014 29271 hybrid_clock.cc:648] HybridClock initialized: now 1786100108241961 us; error 42 us; skew 500 ppm
I20260807 10:55:08.242391 29271 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:55:08 dist-test-slave-x5fp krb5kdc[29105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100108, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:55:08.246356 29271 init.cc:377] Logged in from keytab as kudu/127.3.73.193@KRBTEST.COM (short username kudu)
I20260807 10:55:08.248046 29271 webserver.cc:460] Webserver started at http://127.3.73.193:41591/ using document root <none> and password file <none>
I20260807 10:55:08.248944 29271 fs_manager.cc:359] Metadata directory not provided
I20260807 10:55:08.249114 29271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:55:08.249467 29271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:55:08.253264 29271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-0/data/instance:
uuid: "138ee6b2152b4df9ad228962435d75fc"
format_stamp: "Formatted at 2026-08-07 10:55:08 on dist-test-slave-x5fp"
server_key: "196fc21dff3101a1c8c5f1d1bc389668"
server_key_iv: "85f79671db86823e5d439cafabc9f7f8"
server_key_version: "encryptionkey@0"
I20260807 10:55:08.254194 29271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance:
uuid: "138ee6b2152b4df9ad228962435d75fc"
format_stamp: "Formatted at 2026-08-07 10:55:08 on dist-test-slave-x5fp"
server_key: "196fc21dff3101a1c8c5f1d1bc389668"
server_key_iv: "85f79671db86823e5d439cafabc9f7f8"
server_key_version: "encryptionkey@0"
I20260807 10:55:08.260349 29271 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 10:55:08.265194 29297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:08.266979 29271 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 10:55:08.267220 29271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-0/wal
uuid: "138ee6b2152b4df9ad228962435d75fc"
format_stamp: "Formatted at 2026-08-07 10:55:08 on dist-test-slave-x5fp"
server_key: "196fc21dff3101a1c8c5f1d1bc389668"
server_key_iv: "85f79671db86823e5d439cafabc9f7f8"
server_key_version: "encryptionkey@0"
I20260807 10:55:08.267482 29271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:55:08.294764 29271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:55:08.299917 29271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:55:08.300269 29271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:55:08.301553 29271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:55:08.303279 29271 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:55:08.303392 29271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:08.303704 29271 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:55:08.303843 29271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:08.346848 29271 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.193:39369
I20260807 10:55:08.346953 29472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.193:39369 every 8 connection(s)
I20260807 10:55:08.348308 29271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
I20260807 10:55:08.354483 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 29271
I20260807 10:55:08.354743 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance
I20260807 10:55:08.355477 3367 external_mini_cluster.cc:1428] Setting key 3345e837d51b2b8be2efdbfb9612bc42
Aug 07 10:55:08 dist-test-slave-x5fp krb5kdc[29105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100108, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:55:08.373473 29476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:08.351784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:36869 (local address 127.3.73.254:34889)
0807 10:55:08.352364 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:08.352369 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:08.353658 (+ 1289us) server_negotiation.cc:407] Connection header received
0807 10:55:08.355906 (+ 2248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:08.355912 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:08.356033 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:08.356168 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:08.358486 (+ 2318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:08.359235 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:08.361169 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:08.361630 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:08.361789 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:08.365914 (+ 4125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:08.365920 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:08.365924 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:55:08.365963 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:08.369336 (+ 3373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:08.370518 (+ 1182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:08.370524 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:08.370527 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:08.370676 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:08.371297 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:08.371307 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:08.371310 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:08.371628 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:08.371851 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:08.372807 (+ 956us) server_negotiation.cc:299] Negotiation successful
0807 10:55:08.373086 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":318,"threads_started":1}
I20260807 10:55:08.374985 29477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:08.352513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34889 (local address 127.3.73.193:36869)
0807 10:55:08.353450 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:08.353493 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:08.355511 (+ 2018us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:08.356465 (+ 954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:08.356475 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:08.357541 (+ 1066us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:55:08.358279 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:08.358303 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:08.359425 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:08.359430 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:08.361005 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:08.361020 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:08.361914 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:08.361924 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:08.362179 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:08.362965 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:55:08.362991 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:55:08.365566 (+ 2575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:55:08.369571 (+ 4005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:55:08.369582 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:55:08.369599 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:55:08.370339 (+ 740us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:55:08.370823 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:55:08.370831 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:55:08.370835 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:55:08.371158 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:55:08.371837 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:55:08.371846 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:55:08.372306 (+ 460us) client_negotiation.cc:769] Sending connection context
0807 10:55:08.372634 (+ 328us) client_negotiation.cc:240] Negotiation successful
0807 10:55:08.373291 (+ 657us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"thread_start_us":456,"threads_started":1}
I20260807 10:55:08.377718 29473 heartbeater.cc:347] Connected to a master server at 127.3.73.254:34889
I20260807 10:55:08.378141 29473 heartbeater.cc:464] Registering TS with master...
I20260807 10:55:08.379321 29473 heartbeater.cc:511] Master 127.3.73.254:34889 requested a full tablet report, sending...
I20260807 10:55:08.382074 29182 ts_manager.cc:194] Registered new tserver with Master: 138ee6b2152b4df9ad228962435d75fc (127.3.73.193:39369)
I20260807 10:55:08.383721 29182 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.193@KRBTEST.COM'} at 127.3.73.193:36869
WARNING: no policy specified for kudu/127.3.73.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab.
Entry for principal kudu/127.3.73.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab.
I20260807 10:55:08.451242 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.194:0
--local_ip_for_outbound_sockets=127.3.73.194
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:34889
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:37563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:55:08.843058 29480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:55:08.843462 29480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:55:08.843659 29480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:55:08.852370 29480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:55:08.852571 29480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:55:08.852833 29480 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.194
I20260807 10:55:08.864049 29480 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:37563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:34889
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.194
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:55:08.865567 29480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:55:08.867007 29480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:55:08.882505 29488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:08.882606 29485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:08.882478 29490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:08.883358 29480 server_base.cc:1034] running on GCE node
I20260807 10:55:08.884115 29480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:55:08.885353 29480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:55:08.886615 29480 hybrid_clock.cc:648] HybridClock initialized: now 1786100108886552 us; error 42 us; skew 500 ppm
I20260807 10:55:08.887032 29480 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:55:08 dist-test-slave-x5fp krb5kdc[29105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100108, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:55:08.890985 29480 init.cc:377] Logged in from keytab as kudu/127.3.73.194@KRBTEST.COM (short username kudu)
I20260807 10:55:08.892918 29480 webserver.cc:460] Webserver started at http://127.3.73.194:35705/ using document root <none> and password file <none>
I20260807 10:55:08.893713 29480 fs_manager.cc:359] Metadata directory not provided
I20260807 10:55:08.893838 29480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:55:08.894153 29480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:55:08.897713 29480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-1/data/instance:
uuid: "d58d1b99f00e4dce9afdc3294496e95a"
format_stamp: "Formatted at 2026-08-07 10:55:08 on dist-test-slave-x5fp"
server_key: "5025791f2ea6fb9b548b3217cf5b77d4"
server_key_iv: "d8ceb5058f719b27ba59cc709f8c70f4"
server_key_version: "encryptionkey@0"
I20260807 10:55:08.898609 29480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance:
uuid: "d58d1b99f00e4dce9afdc3294496e95a"
format_stamp: "Formatted at 2026-08-07 10:55:08 on dist-test-slave-x5fp"
server_key: "5025791f2ea6fb9b548b3217cf5b77d4"
server_key_iv: "d8ceb5058f719b27ba59cc709f8c70f4"
server_key_version: "encryptionkey@0"
I20260807 10:55:08.905102 29480 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 10:55:08.909104 29500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:08.910770 29480 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 10:55:08.911054 29480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-1/wal
uuid: "d58d1b99f00e4dce9afdc3294496e95a"
format_stamp: "Formatted at 2026-08-07 10:55:08 on dist-test-slave-x5fp"
server_key: "5025791f2ea6fb9b548b3217cf5b77d4"
server_key_iv: "d8ceb5058f719b27ba59cc709f8c70f4"
server_key_version: "encryptionkey@0"
I20260807 10:55:08.911314 29480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:55:08.931850 29480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:55:08.936674 29480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:55:08.937099 29480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:55:08.938283 29480 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:55:08.939942 29480 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:55:08.940070 29480 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:08.940248 29480 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:55:08.940335 29480 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:08.979902 29480 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.194:43603
I20260807 10:55:08.979960 29678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.194:43603 every 8 connection(s)
I20260807 10:55:08.981575 29480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
I20260807 10:55:08.983314 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 29480
I20260807 10:55:08.983547 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance
I20260807 10:55:08.984200 3367 external_mini_cluster.cc:1428] Setting key 7a0f5335048cd1b17ea1183de5715dfe
Aug 07 10:55:08 dist-test-slave-x5fp krb5kdc[29105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100108, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:55:09.008841 29688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:08.984603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:53337 (local address 127.3.73.254:34889)
0807 10:55:08.985188 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:08.985196 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:08.986516 (+ 1320us) server_negotiation.cc:407] Connection header received
0807 10:55:08.989384 (+ 2868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:08.989389 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:08.989489 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:08.989605 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:08.991999 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:08.993115 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:08.995764 (+ 2649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:08.996581 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:08.996764 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:09.001975 (+ 5211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:09.001985 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:09.001993 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:55:09.002048 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:09.005413 (+ 3365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:09.006309 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:09.006315 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:09.006318 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:09.006457 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:09.006932 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:09.006938 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:09.006941 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:09.007248 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:09.007423 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:09.008162 (+ 739us) server_negotiation.cc:299] Negotiation successful
0807 10:55:09.008414 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":325,"threads_started":1}
I20260807 10:55:09.009950 29689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:08.985334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34889 (local address 127.3.73.194:53337)
0807 10:55:08.986218 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:08.986256 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:08.988986 (+ 2730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:08.989950 (+ 964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:08.989966 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:08.990845 (+ 879us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:55:08.991737 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:08.991768 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:08.993312 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:08.993320 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:08.995566 (+ 2246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:08.995586 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:08.996908 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:08.996919 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:08.997279 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:08.998193 (+ 914us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:55:08.998236 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:55:09.001608 (+ 3372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:55:09.005614 (+ 4006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:55:09.005624 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:55:09.005645 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:55:09.006146 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:55:09.006584 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:55:09.006589 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:55:09.006593 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:55:09.006808 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:55:09.007395 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:55:09.007404 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:55:09.007784 (+ 380us) client_negotiation.cc:769] Sending connection context
0807 10:55:09.008087 (+ 303us) client_negotiation.cc:240] Negotiation successful
0807 10:55:09.008675 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":389,"threads_started":1}
I20260807 10:55:09.011801 29681 heartbeater.cc:347] Connected to a master server at 127.3.73.254:34889
I20260807 10:55:09.012146 29681 heartbeater.cc:464] Registering TS with master...
I20260807 10:55:09.013063 29681 heartbeater.cc:511] Master 127.3.73.254:34889 requested a full tablet report, sending...
I20260807 10:55:09.014771 29182 ts_manager.cc:194] Registered new tserver with Master: d58d1b99f00e4dce9afdc3294496e95a (127.3.73.194:43603)
I20260807 10:55:09.015688 29182 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.194@KRBTEST.COM'} at 127.3.73.194:53337
WARNING: no policy specified for kudu/127.3.73.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab.
Entry for principal kudu/127.3.73.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab.
I20260807 10:55:09.069679 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.195:0
--local_ip_for_outbound_sockets=127.3.73.195
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:34889
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:37563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 10:55:09.388312 29473 heartbeater.cc:503] Master 127.3.73.254:34889 was elected leader, sending a full tablet report...
W20260807 10:55:09.441841 29695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:55:09.442214 29695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:55:09.442394 29695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:55:09.450388 29695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:55:09.450563 29695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:55:09.450780 29695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.195
I20260807 10:55:09.462056 29695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:37563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:34889
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.195
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:55:09.463855 29695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:55:09.465567 29695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:55:09.481084 29701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:09.481055 29704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:09.481153 29702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:09.482934 29695 server_base.cc:1034] running on GCE node
I20260807 10:55:09.483711 29695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:55:09.484843 29695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:55:09.486081 29695 hybrid_clock.cc:648] HybridClock initialized: now 1786100109486004 us; error 65 us; skew 500 ppm
I20260807 10:55:09.486498 29695 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:55:09 dist-test-slave-x5fp krb5kdc[29105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100109, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:55:09.490605 29695 init.cc:377] Logged in from keytab as kudu/127.3.73.195@KRBTEST.COM (short username kudu)
I20260807 10:55:09.492583 29695 webserver.cc:460] Webserver started at http://127.3.73.195:36815/ using document root <none> and password file <none>
I20260807 10:55:09.493604 29695 fs_manager.cc:359] Metadata directory not provided
I20260807 10:55:09.493839 29695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:55:09.494239 29695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:55:09.497808 29695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-2/data/instance:
uuid: "22adb260bfe64abdb2067da38c8be216"
format_stamp: "Formatted at 2026-08-07 10:55:09 on dist-test-slave-x5fp"
server_key: "28c0d7c6bf44cfc5e5993522d8eb6a3f"
server_key_iv: "27c6083ddbebf44465d6b278579107cf"
server_key_version: "encryptionkey@0"
I20260807 10:55:09.498675 29695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance:
uuid: "22adb260bfe64abdb2067da38c8be216"
format_stamp: "Formatted at 2026-08-07 10:55:09 on dist-test-slave-x5fp"
server_key: "28c0d7c6bf44cfc5e5993522d8eb6a3f"
server_key_iv: "27c6083ddbebf44465d6b278579107cf"
server_key_version: "encryptionkey@0"
I20260807 10:55:09.504822 29695 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 10:55:09.509011 29715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:09.510742 29695 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 10:55:09.510941 29695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-2/wal
uuid: "22adb260bfe64abdb2067da38c8be216"
format_stamp: "Formatted at 2026-08-07 10:55:09 on dist-test-slave-x5fp"
server_key: "28c0d7c6bf44cfc5e5993522d8eb6a3f"
server_key_iv: "27c6083ddbebf44465d6b278579107cf"
server_key_version: "encryptionkey@0"
I20260807 10:55:09.511224 29695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:55:09.585268 29695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:55:09.590687 29695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:55:09.591046 29695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:55:09.592224 29695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:55:09.593926 29695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:55:09.594007 29695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:09.594152 29695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:55:09.594206 29695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:09.637315 29695 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.195:45073
I20260807 10:55:09.637379 29873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.195:45073 every 8 connection(s)
I20260807 10:55:09.638952 29695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
I20260807 10:55:09.640940 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 29695
I20260807 10:55:09.641196 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance
I20260807 10:55:09.641705 3367 external_mini_cluster.cc:1428] Setting key 02eafdec956ee5efcfb31f08f2c14015
Aug 07 10:55:09 dist-test-slave-x5fp krb5kdc[29105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100109, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:55:09.665371 29879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:09.642220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:53641 (local address 127.3.73.254:34889)
0807 10:55:09.642888 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:09.642894 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:09.644631 (+ 1737us) server_negotiation.cc:407] Connection header received
0807 10:55:09.647178 (+ 2547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:09.647183 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:09.647276 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:09.647403 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:09.649384 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:09.650356 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:09.652349 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:09.652914 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:09.653032 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:09.657373 (+ 4341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:09.657379 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:09.657382 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:55:09.657425 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:09.661222 (+ 3797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:09.662389 (+ 1167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:09.662395 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:09.662397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:09.662547 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:09.663334 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:09.663338 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:09.663341 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:09.663651 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:09.663829 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:09.664826 (+ 997us) server_negotiation.cc:299] Negotiation successful
0807 10:55:09.665063 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":328,"threads_started":1}
I20260807 10:55:09.666471 29880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:09.642833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:34889 (local address 127.3.73.195:53641)
0807 10:55:09.644116 (+ 1283us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:09.644196 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:09.646764 (+ 2568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:09.647697 (+ 933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:09.647707 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:09.648389 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:55:09.649143 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:09.649165 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:09.650598 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:09.650603 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:09.652160 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:09.652187 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:09.653201 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:09.653215 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:09.653470 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:09.654276 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:55:09.654303 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:55:09.657077 (+ 2774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:55:09.661433 (+ 4356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:55:09.661444 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:55:09.661468 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:55:09.662171 (+ 703us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:55:09.662705 (+ 534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:55:09.662710 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:55:09.662712 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:55:09.663171 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:55:09.663822 (+ 651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:55:09.663829 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:55:09.664238 (+ 409us) client_negotiation.cc:769] Sending connection context
0807 10:55:09.664532 (+ 294us) client_negotiation.cc:240] Negotiation successful
0807 10:55:09.665097 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":787,"thread_start_us":556,"threads_started":1}
I20260807 10:55:09.668393 29874 heartbeater.cc:347] Connected to a master server at 127.3.73.254:34889
I20260807 10:55:09.668931 29874 heartbeater.cc:464] Registering TS with master...
I20260807 10:55:09.669771 29874 heartbeater.cc:511] Master 127.3.73.254:34889 requested a full tablet report, sending...
I20260807 10:55:09.671571 29182 ts_manager.cc:194] Registered new tserver with Master: 22adb260bfe64abdb2067da38c8be216 (127.3.73.195:45073)
I20260807 10:55:09.672595 3367 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 10:55:09.672952 29182 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.195@KRBTEST.COM'} at 127.3.73.195:53641
I20260807 10:55:09.694927 29879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:09.678985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56308 (local address 127.3.73.254:34889)
0807 10:55:09.679189 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:09.679194 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:09.679800 (+ 606us) server_negotiation.cc:407] Connection header received
0807 10:55:09.679873 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:09.679877 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:09.680282 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:09.680462 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:09.682306 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:09.683210 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:09.684925 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:09.685376 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:09.685493 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:09.688815 (+ 3322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:09.688829 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:09.688832 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:55:09.688877 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:09.691837 (+ 2960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:09.692829 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:09.692834 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:09.692837 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:09.692981 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:09.693405 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:09.693409 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:09.693411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:09.693804 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:09.693995 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:09.694446 (+ 451us) server_negotiation.cc:299] Negotiation successful
0807 10:55:09.694640 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260807 10:55:09.699919 29182 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56308:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 10:55:09.707331 29182 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 10:55:09.734671 29901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:09.721731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:39369 (local address 127.0.0.1:54020)
0807 10:55:09.722997 (+ 1266us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:09.723056 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:09.723238 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:09.725914 (+ 2676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:09.725923 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:09.725967 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:55:09.726450 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:09.726468 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:09.728573 (+ 2105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:09.728580 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:09.731261 (+ 2681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:09.731278 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:09.733307 (+ 2029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:09.733314 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:09.733635 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:09.733699 (+ 64us) client_negotiation.cc:769] Sending connection context
0807 10:55:09.733854 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 10:55:09.734096 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":993,"mutex_wait_us":81,"thread_start_us":6084,"threads_started":1}
I20260807 10:55:09.735185 29904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:09.722275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:43603 (local address 127.0.0.1:36366)
0807 10:55:09.724322 (+ 2047us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:09.724365 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:09.724507 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:09.727351 (+ 2844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:09.727358 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:09.727389 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:55:09.727947 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:09.727961 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:09.730005 (+ 2044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:09.730010 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:09.732215 (+ 2205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:09.732239 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:09.734293 (+ 2054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:09.734301 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:09.734500 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:09.734543 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 10:55:09.734618 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 10:55:09.734760 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1944,"mutex_wait_us":1,"thread_start_us":659,"threads_started":1}
I20260807 10:55:09.736122 29902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:09.722265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54020 (local address 127.3.73.193:39369)
0807 10:55:09.723956 (+ 1691us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:09.723969 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:09.725122 (+ 1153us) server_negotiation.cc:407] Connection header received
0807 10:55:09.725268 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:09.725280 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:09.725632 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:09.725887 (+ 255us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:55:09.726678 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:09.728337 (+ 1659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:09.731469 (+ 3132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:09.733068 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:09.733237 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:09.733533 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:09.735569 (+ 2036us) server_negotiation.cc:299] Negotiation successful
0807 10:55:09.735762 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1486,"thread_start_us":1288,"threads_started":1}
I20260807 10:55:09.736420 29903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:09.722777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36366 (local address 127.3.73.194:43603)
0807 10:55:09.726585 (+ 3808us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:09.726597 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:09.726636 (+ 39us) server_negotiation.cc:407] Connection header received
0807 10:55:09.726756 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:09.726767 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:09.727074 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:09.727340 (+ 266us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:55:09.728168 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:09.729833 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:09.732449 (+ 2616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:09.734006 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:09.735620 (+ 1614us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:09.735839 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:09.735940 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 10:55:09.736120 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3649,"thread_start_us":3535,"threads_started":1}
I20260807 10:55:09.737291 29900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:09.723801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:45073 (local address 127.0.0.1:50880)
0807 10:55:09.728152 (+ 4351us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:09.728172 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:09.728323 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:09.729046 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:09.729051 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:09.729081 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:55:09.729609 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:09.729632 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:09.731938 (+ 2306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:09.731944 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:09.734123 (+ 2179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:09.734142 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:09.736089 (+ 1947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:09.736098 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:09.736526 (+ 428us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:09.736562 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 10:55:09.736668 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 10:55:09.736846 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4228,"mutex_wait_us":24,"thread_start_us":1591,"threads_started":1}
I20260807 10:55:09.737663 29905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:09.724415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50880 (local address 127.3.73.195:45073)
0807 10:55:09.725361 (+ 946us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:09.725370 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:09.728316 (+ 2946us) server_negotiation.cc:407] Connection header received
0807 10:55:09.728492 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:09.728503 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:09.728792 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:09.729058 (+ 266us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:55:09.730323 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:09.731778 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:09.734329 (+ 2551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:09.735856 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:09.736004 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:09.736232 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:09.736863 (+ 631us) server_negotiation.cc:299] Negotiation successful
0807 10:55:09.737050 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":722,"thread_start_us":449,"threads_started":1}
I20260807 10:55:09.740641 29578 tablet_service.cc:1467] Processing CreateTablet for tablet 1feba7332f67414bbeba32f7c990c92e (DEFAULT_TABLE table=test-table [id=ebe6225755e1416f8b0c7b5ea11ef4bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:55:09.740643 29379 tablet_service.cc:1467] Processing CreateTablet for tablet 1feba7332f67414bbeba32f7c990c92e (DEFAULT_TABLE table=test-table [id=ebe6225755e1416f8b0c7b5ea11ef4bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:55:09.742138 29578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1feba7332f67414bbeba32f7c990c92e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:55:09.742337 29379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1feba7332f67414bbeba32f7c990c92e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:55:09.741911 29784 tablet_service.cc:1467] Processing CreateTablet for tablet 1feba7332f67414bbeba32f7c990c92e (DEFAULT_TABLE table=test-table [id=ebe6225755e1416f8b0c7b5ea11ef4bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:55:09.743530 29784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1feba7332f67414bbeba32f7c990c92e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:55:09.757294 29906 tablet_bootstrap.cc:492] T 1feba7332f67414bbeba32f7c990c92e P d58d1b99f00e4dce9afdc3294496e95a: Bootstrap starting.
I20260807 10:55:09.762012 29906 tablet_bootstrap.cc:654] T 1feba7332f67414bbeba32f7c990c92e P d58d1b99f00e4dce9afdc3294496e95a: Neither blocks nor log segments found. Creating new log.
I20260807 10:55:09.764425 29907 tablet_bootstrap.cc:492] T 1feba7332f67414bbeba32f7c990c92e P 22adb260bfe64abdb2067da38c8be216: Bootstrap starting.
I20260807 10:55:09.766238 29906 log.cc:824] T 1feba7332f67414bbeba32f7c990c92e P d58d1b99f00e4dce9afdc3294496e95a: Log is configured to *not* fsync() on all Append() calls
I20260807 10:55:09.768780 29907 tablet_bootstrap.cc:654] T 1feba7332f67414bbeba32f7c990c92e P 22adb260bfe64abdb2067da38c8be216: Neither blocks nor log segments found. Creating new log.
I20260807 10:55:09.769984 29909 tablet_bootstrap.cc:492] T 1feba7332f67414bbeba32f7c990c92e P 138ee6b2152b4df9ad228962435d75fc: Bootstrap starting.
I20260807 10:55:09.770382 29906 tablet_bootstrap.cc:492] T 1feba7332f67414bbeba32f7c990c92e P d58d1b99f00e4dce9afdc3294496e95a: No bootstrap required, opened a new log
I20260807 10:55:09.770358 29907 log.cc:824] T 1feba7332f67414bbeba32f7c990c92e P 22adb260bfe64abdb2067da38c8be216: Log is configured to *not* fsync() on all Append() calls
I20260807 10:55:09.770715 29906 ts_tablet_manager.cc:1397] T 1feba7332f67414bbeba32f7c990c92e P d58d1b99f00e4dce9afdc3294496e95a: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.000s
I20260807 10:55:09.774859 29909 tablet_bootstrap.cc:654] T 1feba7332f67414bbeba32f7c990c92e P 138ee6b2152b4df9ad228962435d75fc: Neither blocks nor log segments found. Creating new log.
I20260807 10:55:09.777022 29909 log.cc:824] T 1feba7332f67414bbeba32f7c990c92e P 138ee6b2152b4df9ad228962435d75fc: Log is configured to *not* fsync() on all Append() calls
I20260807 10:55:09.781044 29907 tablet_bootstrap.cc:492] T 1feba7332f67414bbeba32f7c990c92e P 22adb260bfe64abdb2067da38c8be216: No bootstrap required, opened a new log
I20260807 10:55:09.781424 29907 ts_tablet_manager.cc:1397] T 1feba7332f67414bbeba32f7c990c92e P 22adb260bfe64abdb2067da38c8be216: Time spent bootstrapping tablet: real 0.018s user 0.015s sys 0.000s
I20260807 10:55:09.783802 29909 tablet_bootstrap.cc:492] T 1feba7332f67414bbeba32f7c990c92e P 138ee6b2152b4df9ad228962435d75fc: No bootstrap required, opened a new log
I20260807 10:55:09.784169 29909 ts_tablet_manager.cc:1397] T 1feba7332f67414bbeba32f7c990c92e P 138ee6b2152b4df9ad228962435d75fc: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.004s
I20260807 10:55:09.785539 29906 raft_consensus.cc:348] T 1feba7332f67414bbeba32f7c990c92e P d58d1b99f00e4dce9afdc3294496e95a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "138ee6b2152b4df9ad228962435d75fc" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 39369 } } peers { permanent_uuid: "d58d1b99f00e4dce9afdc3294496e95a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43603 } } peers { permanent_uuid: "22adb260bfe64abdb2067da38c8be216" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 45073 } }
I20260807 10:55:09.786105 29906 raft_consensus.cc:374] T 1feba7332f67414bbeba32f7c990c92e P d58d1b99f00e4dce9afdc3294496e95a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:55:09.786222 29906 raft_consensus.cc:729] T 1feba7332f67414bbeba32f7c990c92e P d58d1b99f00e4dce9afdc3294496e95a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d58d1b99f00e4dce9afdc3294496e95a, State: Initialized, Role: FOLLOWER
I20260807 10:55:09.786959 29906 consensus_queue.cc:260] T 1feba7332f67414bbeba32f7c990c92e P d58d1b99f00e4dce9afdc3294496e95a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "138ee6b2152b4df9ad228962435d75fc" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 39369 } } peers { permanent_uuid: "d58d1b99f00e4dce9afdc3294496e95a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43603 } } peers { permanent_uuid: "22adb260bfe64abdb2067da38c8be216" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 45073 } }
I20260807 10:55:09.788457 29681 heartbeater.cc:503] Master 127.3.73.254:34889 was elected leader, sending a full tablet report...
I20260807 10:55:09.789817 29906 ts_tablet_manager.cc:1428] T 1feba7332f67414bbeba32f7c990c92e P d58d1b99f00e4dce9afdc3294496e95a: Time spent starting tablet: real 0.019s user 0.015s sys 0.003s
I20260807 10:55:09.792364 29907 raft_consensus.cc:348] T 1feba7332f67414bbeba32f7c990c92e P 22adb260bfe64abdb2067da38c8be216 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "138ee6b2152b4df9ad228962435d75fc" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 39369 } } peers { permanent_uuid: "d58d1b99f00e4dce9afdc3294496e95a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43603 } } peers { permanent_uuid: "22adb260bfe64abdb2067da38c8be216" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 45073 } }
I20260807 10:55:09.793011 29907 raft_consensus.cc:374] T 1feba7332f67414bbeba32f7c990c92e P 22adb260bfe64abdb2067da38c8be216 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:55:09.793157 29907 raft_consensus.cc:729] T 1feba7332f67414bbeba32f7c990c92e P 22adb260bfe64abdb2067da38c8be216 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22adb260bfe64abdb2067da38c8be216, State: Initialized, Role: FOLLOWER
I20260807 10:55:09.793936 29907 consensus_queue.cc:260] T 1feba7332f67414bbeba32f7c990c92e P 22adb260bfe64abdb2067da38c8be216 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "138ee6b2152b4df9ad228962435d75fc" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 39369 } } peers { permanent_uuid: "d58d1b99f00e4dce9afdc3294496e95a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43603 } } peers { permanent_uuid: "22adb260bfe64abdb2067da38c8be216" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 45073 } }
I20260807 10:55:09.795934 29874 heartbeater.cc:503] Master 127.3.73.254:34889 was elected leader, sending a full tablet report...
I20260807 10:55:09.796115 29907 ts_tablet_manager.cc:1428] T 1feba7332f67414bbeba32f7c990c92e P 22adb260bfe64abdb2067da38c8be216: Time spent starting tablet: real 0.015s user 0.006s sys 0.007s
I20260807 10:55:09.799813 29909 raft_consensus.cc:348] T 1feba7332f67414bbeba32f7c990c92e P 138ee6b2152b4df9ad228962435d75fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "138ee6b2152b4df9ad228962435d75fc" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 39369 } } peers { permanent_uuid: "d58d1b99f00e4dce9afdc3294496e95a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43603 } } peers { permanent_uuid: "22adb260bfe64abdb2067da38c8be216" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 45073 } }
I20260807 10:55:09.800383 29909 raft_consensus.cc:374] T 1feba7332f67414bbeba32f7c990c92e P 138ee6b2152b4df9ad228962435d75fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:55:09.800537 29909 raft_consensus.cc:729] T 1feba7332f67414bbeba32f7c990c92e P 138ee6b2152b4df9ad228962435d75fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 138ee6b2152b4df9ad228962435d75fc, State: Initialized, Role: FOLLOWER
I20260807 10:55:09.801333 29909 consensus_queue.cc:260] T 1feba7332f67414bbeba32f7c990c92e P 138ee6b2152b4df9ad228962435d75fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "138ee6b2152b4df9ad228962435d75fc" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 39369 } } peers { permanent_uuid: "d58d1b99f00e4dce9afdc3294496e95a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43603 } } peers { permanent_uuid: "22adb260bfe64abdb2067da38c8be216" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 45073 } }
I20260807 10:55:09.803525 29909 ts_tablet_manager.cc:1428] T 1feba7332f67414bbeba32f7c990c92e P 138ee6b2152b4df9ad228962435d75fc: Time spent starting tablet: real 0.019s user 0.007s sys 0.007s
W20260807 10:55:09.851502 29474 tablet.cc:2369] T 1feba7332f67414bbeba32f7c990c92e P 138ee6b2152b4df9ad228962435d75fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 10:55:09.891034 29878 tablet.cc:2369] T 1feba7332f67414bbeba32f7c990c92e P 22adb260bfe64abdb2067da38c8be216: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:55:09.978979 29913 raft_consensus.cc:482] T 1feba7332f67414bbeba32f7c990c92e P 22adb260bfe64abdb2067da38c8be216 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 10:55:09.979419 29913 raft_consensus.cc:504] T 1feba7332f67414bbeba32f7c990c92e P 22adb260bfe64abdb2067da38c8be216 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "138ee6b2152b4df9ad228962435d75fc" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 39369 } } peers { permanent_uuid: "d58d1b99f00e4dce9afdc3294496e95a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43603 } } peers { permanent_uuid: "22adb260bfe64abdb2067da38c8be216" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 45073 } }
I20260807 10:55:09.981616 29913 leader_election.cc:288] T 1feba7332f67414bbeba32f7c990c92e P 22adb260bfe64abdb2067da38c8be216 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 138ee6b2152b4df9ad228962435d75fc (127.3.73.193:39369), d58d1b99f00e4dce9afdc3294496e95a (127.3.73.194:43603)
W20260807 10:55:09.984117 29684 tablet.cc:2369] T 1feba7332f67414bbeba32f7c990c92e P d58d1b99f00e4dce9afdc3294496e95a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:55:09.990123 29880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:09.982127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:39369 (local address 127.3.73.195:44501)
0807 10:55:09.982514 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:09.982538 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:09.982698 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:09.983115 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:09.983120 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:09.983180 (+ 60us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:55:09.983463 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:09.983484 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:09.985238 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:09.985245 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:09.987381 (+ 2136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:09.987399 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:09.989228 (+ 1829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:09.989235 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:09.989441 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:09.989475 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 10:55:09.989609 (+ 134us) client_negotiation.cc:240] Negotiation successful
0807 10:55:09.989787 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":32}
I20260807 10:55:09.990108 29902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:09.982322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:44501 (local address 127.3.73.193:39369)
0807 10:55:09.982569 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:09.982578 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:09.982603 (+ 25us) server_negotiation.cc:407] Connection header received
0807 10:55:09.982900 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:09.982904 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:09.982975 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:09.983093 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:55:09.983726 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:09.985046 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:09.987590 (+ 2544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:09.988921 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:09.989130 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:09.989228 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:09.989650 (+ 422us) server_negotiation.cc:299] Negotiation successful
0807 10:55:09.989806 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260807 10:55:09.991539 29916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:09.982784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:43603 (local address 127.3.73.195:51277)
0807 10:55:09.983813 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:09.983836 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:09.984066 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:09.984651 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:09.984656 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:09.984687 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:55:09.985121 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:09.985139 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:09.986339 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:09.986345 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:09.988218 (+ 1873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:09.988236 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:09.990661 (+ 2425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:09.990668 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:09.990858 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:09.990894 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 10:55:09.990993 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 10:55:09.991133 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":891,"mutex_wait_us":1,"thread_start_us":612,"threads_started":1}
I20260807 10:55:09.991550 29903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:09.982374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:51277 (local address 127.3.73.194:43603)
0807 10:55:09.982571 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:09.982577 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:09.983941 (+ 1364us) server_negotiation.cc:407] Connection header received
0807 10:55:09.984260 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:09.984266 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:09.984369 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:09.984442 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:55:09.985351 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:09.986157 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:09.988429 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:09.989957 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:09.990117 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:09.990193 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:09.991067 (+ 874us) server_negotiation.cc:299] Negotiation successful
0807 10:55:09.991239 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 10:55:09.991426 29414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1feba7332f67414bbeba32f7c990c92e" candidate_uuid: "22adb260bfe64abdb2067da38c8be216" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "138ee6b2152b4df9ad228962435d75fc" is_pre_election: true
I20260807 10:55:09.992239 29414 raft_consensus.cc:2454] T 1feba7332f67414bbeba32f7c990c92e P 138ee6b2152b4df9ad228962435d75fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22adb260bfe64abdb2067da38c8be216 in term 0.
I20260807 10:55:09.993192 29717 leader_election.cc:302] T 1feba7332f67414bbeba32f7c990c92e P 22adb260bfe64abdb2067da38c8be216 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 138ee6b2152b4df9ad228962435d75fc, 22adb260bfe64abdb2067da38c8be216; no voters:
I20260807 10:55:09.993052 29617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1feba7332f67414bbeba32f7c990c92e" candidate_uuid: "22adb260bfe64abdb2067da38c8be216" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d58d1b99f00e4dce9afdc3294496e95a" is_pre_election: true
I20260807 10:55:09.993625 29913 raft_consensus.cc:2781] T 1feba7332f67414bbeba32f7c990c92e P 22adb260bfe64abdb2067da38c8be216 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 10:55:09.993606 29617 raft_consensus.cc:2454] T 1feba7332f67414bbeba32f7c990c92e P d58d1b99f00e4dce9afdc3294496e95a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22adb260bfe64abdb2067da38c8be216 in term 0.
I20260807 10:55:09.993803 29913 raft_consensus.cc:482] T 1feba7332f67414bbeba32f7c990c92e P 22adb260bfe64abdb2067da38c8be216 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 10:55:09.993952 29913 raft_consensus.cc:3037] T 1feba7332f67414bbeba32f7c990c92e P 22adb260bfe64abdb2067da38c8be216 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:55:09.996644 29913 raft_consensus.cc:504] T 1feba7332f67414bbeba32f7c990c92e P 22adb260bfe64abdb2067da38c8be216 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "138ee6b2152b4df9ad228962435d75fc" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 39369 } } peers { permanent_uuid: "d58d1b99f00e4dce9afdc3294496e95a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43603 } } peers { permanent_uuid: "22adb260bfe64abdb2067da38c8be216" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 45073 } }
I20260807 10:55:09.997560 29913 leader_election.cc:288] T 1feba7332f67414bbeba32f7c990c92e P 22adb260bfe64abdb2067da38c8be216 [CANDIDATE]: Term 1 election: Requested vote from peers 138ee6b2152b4df9ad228962435d75fc (127.3.73.193:39369), d58d1b99f00e4dce9afdc3294496e95a (127.3.73.194:43603)
I20260807 10:55:09.998099 29414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1feba7332f67414bbeba32f7c990c92e" candidate_uuid: "22adb260bfe64abdb2067da38c8be216" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "138ee6b2152b4df9ad228962435d75fc"
I20260807 10:55:09.998121 29617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1feba7332f67414bbeba32f7c990c92e" candidate_uuid: "22adb260bfe64abdb2067da38c8be216" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d58d1b99f00e4dce9afdc3294496e95a"
I20260807 10:55:09.998360 29617 raft_consensus.cc:3037] T 1feba7332f67414bbeba32f7c990c92e P d58d1b99f00e4dce9afdc3294496e95a [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:55:09.998368 29414 raft_consensus.cc:3037] T 1feba7332f67414bbeba32f7c990c92e P 138ee6b2152b4df9ad228962435d75fc [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:55:10.001097 29617 raft_consensus.cc:2454] T 1feba7332f67414bbeba32f7c990c92e P d58d1b99f00e4dce9afdc3294496e95a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22adb260bfe64abdb2067da38c8be216 in term 1.
I20260807 10:55:10.001330 29414 raft_consensus.cc:2454] T 1feba7332f67414bbeba32f7c990c92e P 138ee6b2152b4df9ad228962435d75fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22adb260bfe64abdb2067da38c8be216 in term 1.
I20260807 10:55:10.001700 29721 leader_election.cc:302] T 1feba7332f67414bbeba32f7c990c92e P 22adb260bfe64abdb2067da38c8be216 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22adb260bfe64abdb2067da38c8be216, d58d1b99f00e4dce9afdc3294496e95a; no voters:
I20260807 10:55:10.002090 29913 raft_consensus.cc:2781] T 1feba7332f67414bbeba32f7c990c92e P 22adb260bfe64abdb2067da38c8be216 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:55:10.002668 29913 raft_consensus.cc:686] T 1feba7332f67414bbeba32f7c990c92e P 22adb260bfe64abdb2067da38c8be216 [term 1 LEADER]: Becoming Leader. State: Replica: 22adb260bfe64abdb2067da38c8be216, State: Running, Role: LEADER
I20260807 10:55:10.003299 29913 consensus_queue.cc:237] T 1feba7332f67414bbeba32f7c990c92e P 22adb260bfe64abdb2067da38c8be216 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "138ee6b2152b4df9ad228962435d75fc" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 39369 } } peers { permanent_uuid: "d58d1b99f00e4dce9afdc3294496e95a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43603 } } peers { permanent_uuid: "22adb260bfe64abdb2067da38c8be216" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 45073 } }
I20260807 10:55:10.008458 29182 catalog_manager.cc:5495] T 1feba7332f67414bbeba32f7c990c92e P 22adb260bfe64abdb2067da38c8be216 reported cstate change: term changed from 0 to 1, leader changed from <none> to 22adb260bfe64abdb2067da38c8be216 (127.3.73.195). New cstate: current_term: 1 leader_uuid: "22adb260bfe64abdb2067da38c8be216" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "138ee6b2152b4df9ad228962435d75fc" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 39369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d58d1b99f00e4dce9afdc3294496e95a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22adb260bfe64abdb2067da38c8be216" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 45073 } health_report { overall_health: HEALTHY } } }
I20260807 10:55:10.037204 29905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:10.031283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50892 (local address 127.3.73.195:45073)
0807 10:55:10.031556 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:10.031565 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:10.031621 (+ 56us) server_negotiation.cc:407] Connection header received
0807 10:55:10.031824 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:10.031829 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:10.031932 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:10.032094 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:55:10.032488 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:10.033350 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:10.034859 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:10.035303 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:10.035409 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:10.035968 (+ 559us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:55:10.036518 (+ 550us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:55:10.036631 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:10.036813 (+ 182us) server_negotiation.cc:299] Negotiation successful
0807 10:55:10.036888 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260807 10:55:10.046125 29617 raft_consensus.cc:1261] T 1feba7332f67414bbeba32f7c990c92e P d58d1b99f00e4dce9afdc3294496e95a [term 1 FOLLOWER]: Refusing update from remote peer 22adb260bfe64abdb2067da38c8be216: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:55:10.046126 29414 raft_consensus.cc:1261] T 1feba7332f67414bbeba32f7c990c92e P 138ee6b2152b4df9ad228962435d75fc [term 1 FOLLOWER]: Refusing update from remote peer 22adb260bfe64abdb2067da38c8be216: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:55:10.047324 29917 consensus_queue.cc:1035] T 1feba7332f67414bbeba32f7c990c92e P 22adb260bfe64abdb2067da38c8be216 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d58d1b99f00e4dce9afdc3294496e95a" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 43603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:55:10.047724 29913 consensus_queue.cc:1035] T 1feba7332f67414bbeba32f7c990c92e P 22adb260bfe64abdb2067da38c8be216 [LEADER]: Connected to new peer: Peer: permanent_uuid: "138ee6b2152b4df9ad228962435d75fc" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 39369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:55:10.093439 29181 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:55:10.093762 29181 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:55:10.098160 29181 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7ba6a47e245f47ad8e7fd7315430f9a6: Sending DeleteTablet for 3 replicas of tablet 1feba7332f67414bbeba32f7c990c92e
I20260807 10:55:10.099711 29578 tablet_service.cc:1514] Processing DeleteTablet for tablet 1feba7332f67414bbeba32f7c990c92e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:55:10 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:36366
I20260807 10:55:10.099684 29379 tablet_service.cc:1514] Processing DeleteTablet for tablet 1feba7332f67414bbeba32f7c990c92e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:55:10 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:54020
I20260807 10:55:10.100054 29784 tablet_service.cc:1514] Processing DeleteTablet for tablet 1feba7332f67414bbeba32f7c990c92e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:55:10 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:50880
I20260807 10:55:10.105506 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 29271
I20260807 10:55:10.112236 29944 tablet_replica.cc:330] T 1feba7332f67414bbeba32f7c990c92e P 138ee6b2152b4df9ad228962435d75fc: stopping tablet replica
I20260807 10:55:10.113062 29944 raft_consensus.cc:2229] T 1feba7332f67414bbeba32f7c990c92e P 138ee6b2152b4df9ad228962435d75fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:55:10.113865 29944 raft_consensus.cc:2258] T 1feba7332f67414bbeba32f7c990c92e P 138ee6b2152b4df9ad228962435d75fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:55:10.116698 29944 ts_tablet_manager.cc:1905] T 1feba7332f67414bbeba32f7c990c92e P 138ee6b2152b4df9ad228962435d75fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:55:10.121433 29945 tablet_replica.cc:330] T 1feba7332f67414bbeba32f7c990c92e P 22adb260bfe64abdb2067da38c8be216: stopping tablet replica
I20260807 10:55:10.123224 29945 raft_consensus.cc:2229] T 1feba7332f67414bbeba32f7c990c92e P 22adb260bfe64abdb2067da38c8be216 [term 1 LEADER]: Raft consensus shutting down.
I20260807 10:55:10.124089 29945 raft_consensus.cc:2258] T 1feba7332f67414bbeba32f7c990c92e P 22adb260bfe64abdb2067da38c8be216 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:55:10.126106 29944 ts_tablet_manager.cc:1918] T 1feba7332f67414bbeba32f7c990c92e P 138ee6b2152b4df9ad228962435d75fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:55:10.126398 29944 log.cc:1196] T 1feba7332f67414bbeba32f7c990c92e P 138ee6b2152b4df9ad228962435d75fc: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-0/wal/wals/1feba7332f67414bbeba32f7c990c92e
I20260807 10:55:10.127242 29944 ts_tablet_manager.cc:1939] T 1feba7332f67414bbeba32f7c990c92e P 138ee6b2152b4df9ad228962435d75fc: Deleting consensus metadata
I20260807 10:55:10.130067 29157 catalog_manager.cc:4841] TS 138ee6b2152b4df9ad228962435d75fc (127.3.73.193:39369): tablet 1feba7332f67414bbeba32f7c990c92e (table test-table [id=ebe6225755e1416f8b0c7b5ea11ef4bd]) successfully deleted
I20260807 10:55:10.131317 29940 tablet_replica.cc:330] T 1feba7332f67414bbeba32f7c990c92e P d58d1b99f00e4dce9afdc3294496e95a: stopping tablet replica
I20260807 10:55:10.131855 29940 raft_consensus.cc:2229] T 1feba7332f67414bbeba32f7c990c92e P d58d1b99f00e4dce9afdc3294496e95a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:55:10.132514 29940 raft_consensus.cc:2258] T 1feba7332f67414bbeba32f7c990c92e P d58d1b99f00e4dce9afdc3294496e95a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:55:10.135736 29940 ts_tablet_manager.cc:1905] T 1feba7332f67414bbeba32f7c990c92e P d58d1b99f00e4dce9afdc3294496e95a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:55:10.137418 29945 ts_tablet_manager.cc:1905] T 1feba7332f67414bbeba32f7c990c92e P 22adb260bfe64abdb2067da38c8be216: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 07 10:55:10 dist-test-slave-x5fp krb5kdc[29105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.193@KRBTEST.COM
I20260807 10:55:10.143564 29940 ts_tablet_manager.cc:1918] T 1feba7332f67414bbeba32f7c990c92e P d58d1b99f00e4dce9afdc3294496e95a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:55:10.144418 29940 log.cc:1196] T 1feba7332f67414bbeba32f7c990c92e P d58d1b99f00e4dce9afdc3294496e95a: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-1/wal/wals/1feba7332f67414bbeba32f7c990c92e
I20260807 10:55:10.145110 29940 ts_tablet_manager.cc:1939] T 1feba7332f67414bbeba32f7c990c92e P d58d1b99f00e4dce9afdc3294496e95a: Deleting consensus metadata
I20260807 10:55:10.145717 29945 ts_tablet_manager.cc:1918] T 1feba7332f67414bbeba32f7c990c92e P 22adb260bfe64abdb2067da38c8be216: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:55:10.145939 29945 log.cc:1196] T 1feba7332f67414bbeba32f7c990c92e P 22adb260bfe64abdb2067da38c8be216: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786099767749326-3367-0/minicluster-data/ts-2/wal/wals/1feba7332f67414bbeba32f7c990c92e
I20260807 10:55:10.146612 29945 ts_tablet_manager.cc:1939] T 1feba7332f67414bbeba32f7c990c92e P 22adb260bfe64abdb2067da38c8be216: Deleting consensus metadata
I20260807 10:55:10.147621 29161 catalog_manager.cc:4841] TS d58d1b99f00e4dce9afdc3294496e95a (127.3.73.194:43603): tablet 1feba7332f67414bbeba32f7c990c92e (table test-table [id=ebe6225755e1416f8b0c7b5ea11ef4bd]) successfully deleted
I20260807 10:55:10.148460 29161 catalog_manager.cc:4841] TS 22adb260bfe64abdb2067da38c8be216 (127.3.73.195:45073): tablet 1feba7332f67414bbeba32f7c990c92e (table test-table [id=ebe6225755e1416f8b0c7b5ea11ef4bd]) successfully deleted
I20260807 10:55:10.149922 29902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:10.107375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54026 (local address 127.3.73.193:39369)
0807 10:55:10.107600 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:10.107609 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:10.110281 (+ 2672us) server_negotiation.cc:407] Connection header received
0807 10:55:10.111354 (+ 1073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:10.111364 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:10.111525 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:10.111644 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:10.121700 (+ 10056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:10.123358 (+ 1658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:10.126449 (+ 3091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:10.127273 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:10.127391 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:10.139518 (+ 12127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:10.139545 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:10.139566 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:55:10.139638 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:10.145059 (+ 5421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:10.146550 (+ 1491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:10.146564 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:10.146570 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:10.146799 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:10.147368 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:10.147375 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:10.147378 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:10.147941 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:10.148287 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:10.149084 (+ 797us) server_negotiation.cc:299] Negotiation successful
0807 10:55:10.149376 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 10:55:10.151081 29468 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:55:10.254070 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 29271
I20260807 10:55:10.288647 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 29480
Aug 07 10:55:10 dist-test-slave-x5fp krb5kdc[29105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.194@KRBTEST.COM
I20260807 10:55:10.311215 29903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:10.289716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37724 (local address 127.3.73.194:43603)
0807 10:55:10.289950 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:10.289958 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:10.289982 (+ 24us) server_negotiation.cc:407] Connection header received
0807 10:55:10.290054 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:10.290062 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:10.290164 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:10.290297 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:10.291947 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:10.293449 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:10.295080 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:10.295784 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:10.295924 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:10.301304 (+ 5380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:10.301328 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:10.301350 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:55:10.301407 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:10.306163 (+ 4756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:10.307460 (+ 1297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:10.307472 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:10.307476 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:10.307683 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:10.308432 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:10.308442 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:10.308446 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:10.309434 (+ 988us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:10.309905 (+ 471us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:10.310437 (+ 532us) server_negotiation.cc:299] Negotiation successful
0807 10:55:10.310762 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 10:55:10.312114 29671 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:55:10.446602 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 29480
I20260807 10:55:10.475203 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 29695
Aug 07 10:55:10 dist-test-slave-x5fp krb5kdc[29105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.195@KRBTEST.COM
I20260807 10:55:10.497200 29905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:10.476118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50340 (local address 127.3.73.195:45073)
0807 10:55:10.476327 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:10.476332 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:10.476476 (+ 144us) server_negotiation.cc:407] Connection header received
0807 10:55:10.476653 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:10.476658 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:10.476867 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:10.477013 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:10.478982 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:10.479854 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:10.482028 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:10.482539 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:10.482654 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:10.488767 (+ 6113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:10.488790 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:10.488814 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:55:10.488871 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:10.492890 (+ 4019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:10.493874 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:10.493890 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:10.493894 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:10.494411 (+ 517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:10.495231 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:10.495238 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:10.495242 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:10.495713 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:10.496175 (+ 462us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:10.496456 (+ 281us) server_negotiation.cc:299] Negotiation successful
0807 10:55:10.496767 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 10:55:10.498097 29866 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:55:10.635677 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 29695
I20260807 10:55:10.665944 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 29126
I20260807 10:55:10.667512 29231 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:55:10.787338 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 29126
2026-08-07T10:55:10Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (5551 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T10:55:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:55:10Z Disabled control of system clock
I20260807 10:55:10.909161 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:44115
--webserver_interface=127.3.73.254
--webserver_port=0
--builtin_ntp_servers=127.3.73.212:40185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:44115
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 10:55:11.292598 29958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:55:11.293045 29958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:55:11.293138 29958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:55:11.301085 29958 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:55:11.301227 29958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:55:11.301266 29958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:55:11.301321 29958 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:55:11.301366 29958 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:55:11.311249 29958 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:40185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:44115
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:44115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:55:11.312875 29958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:55:11.314859 29958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:55:11.328711 29963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:11.329157 29966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:11.328704 29964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:11.330677 29958 server_base.cc:1034] running on GCE node
I20260807 10:55:11.331641 29958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:55:11.333413 29958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:55:11.334682 29958 hybrid_clock.cc:648] HybridClock initialized: now 1786100111334635 us; error 33 us; skew 500 ppm
I20260807 10:55:11.335100 29958 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 10:55:11.338111 29958 webserver.cc:460] Webserver started at http://127.3.73.254:37497/ using document root <none> and password file <none>
I20260807 10:55:11.338941 29958 fs_manager.cc:359] Metadata directory not provided
I20260807 10:55:11.339048 29958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:55:11.339380 29958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:55:11.342965 29958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "e13c5e492fa746aab2cee50be5b626b8"
format_stamp: "Formatted at 2026-08-07 10:55:11 on dist-test-slave-x5fp"
server_key: "b127594c122ffea7a5db3b0304274098"
server_key_iv: "57c3c14108e13265bf1a6f5cd5ea4ace"
server_key_version: "encryptionkey@0"
I20260807 10:55:11.343916 29958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "e13c5e492fa746aab2cee50be5b626b8"
format_stamp: "Formatted at 2026-08-07 10:55:11 on dist-test-slave-x5fp"
server_key: "b127594c122ffea7a5db3b0304274098"
server_key_iv: "57c3c14108e13265bf1a6f5cd5ea4ace"
server_key_version: "encryptionkey@0"
I20260807 10:55:11.351011 29958 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.003s
I20260807 10:55:11.355628 29972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:11.357616 29958 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260807 10:55:11.357836 29958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "e13c5e492fa746aab2cee50be5b626b8"
format_stamp: "Formatted at 2026-08-07 10:55:11 on dist-test-slave-x5fp"
server_key: "b127594c122ffea7a5db3b0304274098"
server_key_iv: "57c3c14108e13265bf1a6f5cd5ea4ace"
server_key_version: "encryptionkey@0"
I20260807 10:55:11.358081 29958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:55:11.380901 29958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:55:11.381858 29958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:55:11.382143 29958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:55:11.407305 29958 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:44115
I20260807 10:55:11.407323 30044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:44115 every 8 connection(s)
I20260807 10:55:11.409071 29958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:55:11.413761 30045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:55:11.418993 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 29958
I20260807 10:55:11.419209 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:55:11.419821 3367 external_mini_cluster.cc:1428] Setting key 9b0d73663805d48d8ff111292e0d6ab2
I20260807 10:55:11.426820 30045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e13c5e492fa746aab2cee50be5b626b8: Bootstrap starting.
I20260807 10:55:11.431020 30045 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e13c5e492fa746aab2cee50be5b626b8: Neither blocks nor log segments found. Creating new log.
I20260807 10:55:11.432471 30045 log.cc:824] T 00000000000000000000000000000000 P e13c5e492fa746aab2cee50be5b626b8: Log is configured to *not* fsync() on all Append() calls
I20260807 10:55:11.436429 30045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e13c5e492fa746aab2cee50be5b626b8: No bootstrap required, opened a new log
I20260807 10:55:11.436769 30055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:11.422031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50122 (local address 127.3.73.254:44115)
0807 10:55:11.423194 (+ 1163us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:11.423210 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:11.423259 (+ 49us) server_negotiation.cc:407] Connection header received
0807 10:55:11.425257 (+ 1998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:11.425285 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:11.425816 (+ 531us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:11.426198 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:11.426948 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:11.428360 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:11.430591 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:11.431630 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:11.431826 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:11.433507 (+ 1681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:11.433526 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:11.433549 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:11.434421 (+ 872us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:11.434519 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:11.434545 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:11.434652 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:11.435011 (+ 359us) server_negotiation.cc:299] Negotiation successful
0807 10:55:11.435472 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":680,"thread_start_us":443,"threads_started":1}
I20260807 10:55:11.445057 30045 raft_consensus.cc:348] T 00000000000000000000000000000000 P e13c5e492fa746aab2cee50be5b626b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e13c5e492fa746aab2cee50be5b626b8" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 44115 } }
I20260807 10:55:11.445510 30045 raft_consensus.cc:374] T 00000000000000000000000000000000 P e13c5e492fa746aab2cee50be5b626b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:55:11.445626 30045 raft_consensus.cc:729] T 00000000000000000000000000000000 P e13c5e492fa746aab2cee50be5b626b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e13c5e492fa746aab2cee50be5b626b8, State: Initialized, Role: FOLLOWER
I20260807 10:55:11.446352 30045 consensus_queue.cc:260] T 00000000000000000000000000000000 P e13c5e492fa746aab2cee50be5b626b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e13c5e492fa746aab2cee50be5b626b8" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 44115 } }
I20260807 10:55:11.446663 30045 raft_consensus.cc:388] T 00000000000000000000000000000000 P e13c5e492fa746aab2cee50be5b626b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:55:11.446825 30045 raft_consensus.cc:482] T 00000000000000000000000000000000 P e13c5e492fa746aab2cee50be5b626b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:55:11.447048 30045 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e13c5e492fa746aab2cee50be5b626b8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:55:11.449507 30045 raft_consensus.cc:504] T 00000000000000000000000000000000 P e13c5e492fa746aab2cee50be5b626b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e13c5e492fa746aab2cee50be5b626b8" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 44115 } }
I20260807 10:55:11.450109 30045 leader_election.cc:302] T 00000000000000000000000000000000 P e13c5e492fa746aab2cee50be5b626b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e13c5e492fa746aab2cee50be5b626b8; no voters:
I20260807 10:55:11.450814 30045 leader_election.cc:288] T 00000000000000000000000000000000 P e13c5e492fa746aab2cee50be5b626b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:55:11.451022 30057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e13c5e492fa746aab2cee50be5b626b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:55:11.452183 30057 raft_consensus.cc:686] T 00000000000000000000000000000000 P e13c5e492fa746aab2cee50be5b626b8 [term 1 LEADER]: Becoming Leader. State: Replica: e13c5e492fa746aab2cee50be5b626b8, State: Running, Role: LEADER
I20260807 10:55:11.453038 30057 consensus_queue.cc:237] T 00000000000000000000000000000000 P e13c5e492fa746aab2cee50be5b626b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e13c5e492fa746aab2cee50be5b626b8" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 44115 } }
I20260807 10:55:11.453644 30045 sys_catalog.cc:564] T 00000000000000000000000000000000 P e13c5e492fa746aab2cee50be5b626b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:55:11.456501 30059 sys_catalog.cc:455] T 00000000000000000000000000000000 P e13c5e492fa746aab2cee50be5b626b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e13c5e492fa746aab2cee50be5b626b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e13c5e492fa746aab2cee50be5b626b8" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 44115 } } }
I20260807 10:55:11.456980 30059 sys_catalog.cc:458] T 00000000000000000000000000000000 P e13c5e492fa746aab2cee50be5b626b8 [sys.catalog]: This master's current role is: LEADER
I20260807 10:55:11.456619 30062 sys_catalog.cc:455] T 00000000000000000000000000000000 P e13c5e492fa746aab2cee50be5b626b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e13c5e492fa746aab2cee50be5b626b8. Latest consensus state: current_term: 1 leader_uuid: "e13c5e492fa746aab2cee50be5b626b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e13c5e492fa746aab2cee50be5b626b8" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 44115 } } }
I20260807 10:55:11.457103 30062 sys_catalog.cc:458] T 00000000000000000000000000000000 P e13c5e492fa746aab2cee50be5b626b8 [sys.catalog]: This master's current role is: LEADER
I20260807 10:55:11.459029 30071 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:55:11.465251 30071 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:55:11.476264 30071 catalog_manager.cc:1347] Generated new cluster ID: b75a29111b5c465aaf2c67fa279e4783
I20260807 10:55:11.476472 30071 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:55:11.502920 30071 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:55:11.504537 30071 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:55:11.518494 30071 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e13c5e492fa746aab2cee50be5b626b8: Generated new TSK 0
I20260807 10:55:11.519579 30071 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 10:55:11.532013 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.193:0
--local_ip_for_outbound_sockets=127.3.73.193
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:44115
--builtin_ntp_servers=127.3.73.212:40185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 10:55:11.957441 30089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:55:11.957912 30089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:55:11.958118 30089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:55:11.969259 30089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:55:11.969410 30089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:55:11.969610 30089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.193
I20260807 10:55:11.980965 30089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:40185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:44115
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.193
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:55:11.982543 30089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:55:11.984694 30089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:55:12.001009 30096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:12.001022 30095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:12.001274 30098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:12.001981 30089 server_base.cc:1034] running on GCE node
I20260807 10:55:12.002696 30089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:55:12.003824 30089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:55:12.005126 30089 hybrid_clock.cc:648] HybridClock initialized: now 1786100112005052 us; error 75 us; skew 500 ppm
I20260807 10:55:12.005599 30089 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 10:55:12.008488 30089 webserver.cc:460] Webserver started at http://127.3.73.193:40087/ using document root <none> and password file <none>
I20260807 10:55:12.009380 30089 fs_manager.cc:359] Metadata directory not provided
I20260807 10:55:12.009500 30089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:55:12.009850 30089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:55:12.013491 30089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-0/data/instance:
uuid: "7cf3d5f3af9a4c2387a687b65feb11ff"
format_stamp: "Formatted at 2026-08-07 10:55:12 on dist-test-slave-x5fp"
server_key: "fd8a78c4c00080a26bf90668d2bfac46"
server_key_iv: "4575e993acd16d78b1271174b39fa5af"
server_key_version: "encryptionkey@0"
I20260807 10:55:12.014444 30089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance:
uuid: "7cf3d5f3af9a4c2387a687b65feb11ff"
format_stamp: "Formatted at 2026-08-07 10:55:12 on dist-test-slave-x5fp"
server_key: "fd8a78c4c00080a26bf90668d2bfac46"
server_key_iv: "4575e993acd16d78b1271174b39fa5af"
server_key_version: "encryptionkey@0"
I20260807 10:55:12.021320 30089 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 10:55:12.025467 30106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:12.027275 30089 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 10:55:12.027535 30089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-0/wal
uuid: "7cf3d5f3af9a4c2387a687b65feb11ff"
format_stamp: "Formatted at 2026-08-07 10:55:12 on dist-test-slave-x5fp"
server_key: "fd8a78c4c00080a26bf90668d2bfac46"
server_key_iv: "4575e993acd16d78b1271174b39fa5af"
server_key_version: "encryptionkey@0"
I20260807 10:55:12.027829 30089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:55:12.065798 30089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:55:12.066781 30089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:55:12.067129 30089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:55:12.068272 30089 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:55:12.069983 30089 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:55:12.070115 30089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:12.070276 30089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:55:12.070371 30089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:12.107599 30089 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.193:42111
I20260807 10:55:12.107713 30272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.193:42111 every 8 connection(s)
I20260807 10:55:12.109086 30089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
I20260807 10:55:12.109485 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 30089
I20260807 10:55:12.109684 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance
I20260807 10:55:12.110468 3367 external_mini_cluster.cc:1428] Setting key d7a052eeea2aaa8841d32c42f895866c
I20260807 10:55:12.114251 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.194:0
--local_ip_for_outbound_sockets=127.3.73.194
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:44115
--builtin_ntp_servers=127.3.73.212:40185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 10:55:12.123835 30279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:12.112266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:37927 (local address 127.3.73.254:44115)
0807 10:55:12.112828 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:12.112835 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:12.114349 (+ 1514us) server_negotiation.cc:407] Connection header received
0807 10:55:12.116378 (+ 2029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:12.116385 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:12.116492 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:12.116625 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:12.117831 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:12.118575 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:12.120270 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:12.120687 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:12.120880 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:12.122837 (+ 1957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:12.122843 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:12.122846 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:12.122867 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:12.122919 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:12.122931 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:12.123013 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:12.123347 (+ 334us) server_negotiation.cc:299] Negotiation successful
0807 10:55:12.123485 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":254,"threads_started":1}
I20260807 10:55:12.125398 30280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:12.113058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:44115 (local address 127.3.73.193:37927)
0807 10:55:12.114160 (+ 1102us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:12.114206 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:12.116049 (+ 1843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:12.116915 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:12.116924 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:12.117019 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:55:12.117636 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:12.117655 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:12.118772 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:12.118776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:12.120098 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:12.120111 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:12.121008 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:12.121019 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:12.121403 (+ 384us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:12.122344 (+ 941us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 10:55:12.122350 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:55:12.122415 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 10:55:12.122548 (+ 133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:55:12.123047 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:55:12.123062 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:55:12.123074 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 10:55:12.123295 (+ 221us) client_negotiation.cc:240] Negotiation successful
0807 10:55:12.123784 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":392,"threads_started":1}
I20260807 10:55:12.127169 30274 heartbeater.cc:347] Connected to a master server at 127.3.73.254:44115
I20260807 10:55:12.127538 30274 heartbeater.cc:464] Registering TS with master...
I20260807 10:55:12.128358 30274 heartbeater.cc:511] Master 127.3.73.254:44115 requested a full tablet report, sending...
I20260807 10:55:12.131099 29994 ts_manager.cc:194] Registered new tserver with Master: 7cf3d5f3af9a4c2387a687b65feb11ff (127.3.73.193:42111)
I20260807 10:55:12.133023 29994 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.73.193:37927
W20260807 10:55:12.494954 30283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:55:12.495261 30283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:55:12.495352 30283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:55:12.503403 30283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:55:12.503541 30283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:55:12.503656 30283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.194
I20260807 10:55:12.513469 30283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:40185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:44115
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.194
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:55:12.514837 30283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:55:12.516769 30283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:55:12.530678 30290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:12.531553 30291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:12.532920 30294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:12.533497 30283 server_base.cc:1034] running on GCE node
I20260807 10:55:12.534409 30283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:55:12.535948 30283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:55:12.537245 30283 hybrid_clock.cc:648] HybridClock initialized: now 1786100112537137 us; error 99 us; skew 500 ppm
I20260807 10:55:12.537636 30283 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 10:55:12.540309 30283 webserver.cc:460] Webserver started at http://127.3.73.194:42485/ using document root <none> and password file <none>
I20260807 10:55:12.541213 30283 fs_manager.cc:359] Metadata directory not provided
I20260807 10:55:12.541368 30283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:55:12.541750 30283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:55:12.545182 30283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-1/data/instance:
uuid: "0205eed7f4c24db5963189e4391818af"
format_stamp: "Formatted at 2026-08-07 10:55:12 on dist-test-slave-x5fp"
server_key: "7ca14dfbfd68046e3a273ee02435cfe1"
server_key_iv: "e66fec863d4599466cff43f3a9787358"
server_key_version: "encryptionkey@0"
I20260807 10:55:12.546128 30283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance:
uuid: "0205eed7f4c24db5963189e4391818af"
format_stamp: "Formatted at 2026-08-07 10:55:12 on dist-test-slave-x5fp"
server_key: "7ca14dfbfd68046e3a273ee02435cfe1"
server_key_iv: "e66fec863d4599466cff43f3a9787358"
server_key_version: "encryptionkey@0"
I20260807 10:55:12.552547 30283 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.003s
I20260807 10:55:12.557487 30302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:12.559273 30283 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 10:55:12.559520 30283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-1/wal
uuid: "0205eed7f4c24db5963189e4391818af"
format_stamp: "Formatted at 2026-08-07 10:55:12 on dist-test-slave-x5fp"
server_key: "7ca14dfbfd68046e3a273ee02435cfe1"
server_key_iv: "e66fec863d4599466cff43f3a9787358"
server_key_version: "encryptionkey@0"
I20260807 10:55:12.559878 30283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:55:12.579124 30283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:55:12.580067 30283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:55:12.580374 30283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:55:12.581521 30283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:55:12.583144 30283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:55:12.583243 30283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:12.583320 30283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:55:12.583377 30283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:12.623142 30283 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.194:39411
I20260807 10:55:12.623194 30463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.194:39411 every 8 connection(s)
I20260807 10:55:12.625392 30283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
I20260807 10:55:12.634979 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 30283
I20260807 10:55:12.635185 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance
I20260807 10:55:12.635891 3367 external_mini_cluster.cc:1428] Setting key 568b67d1d7422e44100d14ca0e1fe5cb
I20260807 10:55:12.639468 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.195:0
--local_ip_for_outbound_sockets=127.3.73.195
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:44115
--builtin_ntp_servers=127.3.73.212:40185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 10:55:12.640941 30466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:12.628615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:60703 (local address 127.3.73.254:44115)
0807 10:55:12.629147 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:12.629152 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:12.630488 (+ 1336us) server_negotiation.cc:407] Connection header received
0807 10:55:12.632785 (+ 2297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:12.632791 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:12.632894 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:12.633049 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:12.634281 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:12.635163 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:12.637591 (+ 2428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:12.638242 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:12.638366 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:12.640072 (+ 1706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:12.640077 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:12.640081 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:12.640099 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:12.640142 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:12.640154 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:12.640221 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:12.640504 (+ 283us) server_negotiation.cc:299] Negotiation successful
0807 10:55:12.640593 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":306,"threads_started":1}
I20260807 10:55:12.642674 30468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:12.629180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:44115 (local address 127.3.73.194:60703)
0807 10:55:12.630286 (+ 1106us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:12.630339 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:12.632375 (+ 2036us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:12.633324 (+ 949us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:12.633333 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:12.633432 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:55:12.634080 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:12.634100 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:12.635410 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:12.635416 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:12.637429 (+ 2013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:12.637444 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:12.638521 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:12.638531 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:12.638842 (+ 311us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:12.639658 (+ 816us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 10:55:12.639664 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:55:12.639729 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 10:55:12.639846 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:55:12.640254 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:55:12.640267 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:55:12.640278 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 10:55:12.640490 (+ 212us) client_negotiation.cc:240] Negotiation successful
0807 10:55:12.641015 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":383,"threads_started":1}
I20260807 10:55:12.644433 30464 heartbeater.cc:347] Connected to a master server at 127.3.73.254:44115
I20260807 10:55:12.644865 30464 heartbeater.cc:464] Registering TS with master...
I20260807 10:55:12.645718 30464 heartbeater.cc:511] Master 127.3.73.254:44115 requested a full tablet report, sending...
I20260807 10:55:12.647526 29994 ts_manager.cc:194] Registered new tserver with Master: 0205eed7f4c24db5963189e4391818af (127.3.73.194:39411)
I20260807 10:55:12.648699 29994 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.73.194:60703
I20260807 10:55:13.137627 30274 heartbeater.cc:503] Master 127.3.73.254:44115 was elected leader, sending a full tablet report...
W20260807 10:55:13.141469 30469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:55:13.141990 30469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:55:13.142289 30469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:55:13.155223 30469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:55:13.155382 30469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:55:13.155663 30469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.195
I20260807 10:55:13.171087 30469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:40185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:44115
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.195
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:55:13.173399 30469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:55:13.176777 30469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:55:13.201381 30478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:13.201285 30480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:13.201272 30482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:13.202935 30469 server_base.cc:1034] running on GCE node
I20260807 10:55:13.204105 30469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:55:13.205950 30469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:55:13.207494 30469 hybrid_clock.cc:648] HybridClock initialized: now 1786100113207307 us; error 114 us; skew 500 ppm
I20260807 10:55:13.208170 30469 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 10:55:13.212563 30469 webserver.cc:460] Webserver started at http://127.3.73.195:40603/ using document root <none> and password file <none>
I20260807 10:55:13.213932 30469 fs_manager.cc:359] Metadata directory not provided
I20260807 10:55:13.214148 30469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:55:13.214716 30469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:55:13.219928 30469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-2/data/instance:
uuid: "fcb702762962400783361d3faf740d63"
format_stamp: "Formatted at 2026-08-07 10:55:13 on dist-test-slave-x5fp"
server_key: "6c649a6ab829142989a6f2ded9a227db"
server_key_iv: "4ed5ca0f3a025e2fe26a444c83b9c8df"
server_key_version: "encryptionkey@0"
I20260807 10:55:13.221531 30469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance:
uuid: "fcb702762962400783361d3faf740d63"
format_stamp: "Formatted at 2026-08-07 10:55:13 on dist-test-slave-x5fp"
server_key: "6c649a6ab829142989a6f2ded9a227db"
server_key_iv: "4ed5ca0f3a025e2fe26a444c83b9c8df"
server_key_version: "encryptionkey@0"
I20260807 10:55:13.231654 30469 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.004s sys 0.004s
I20260807 10:55:13.238988 30491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:13.242134 30469 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.003s
I20260807 10:55:13.242452 30469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-2/wal
uuid: "fcb702762962400783361d3faf740d63"
format_stamp: "Formatted at 2026-08-07 10:55:13 on dist-test-slave-x5fp"
server_key: "6c649a6ab829142989a6f2ded9a227db"
server_key_iv: "4ed5ca0f3a025e2fe26a444c83b9c8df"
server_key_version: "encryptionkey@0"
I20260807 10:55:13.242816 30469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:55:13.279012 30469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:55:13.280514 30469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:55:13.281189 30469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:55:13.282980 30469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:55:13.285738 30469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:55:13.285895 30469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:13.286140 30469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:55:13.286302 30469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:13.346143 30469 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.195:36965
I20260807 10:55:13.346300 30658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.195:36965 every 8 connection(s)
I20260807 10:55:13.348433 30469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
I20260807 10:55:13.349334 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 30469
I20260807 10:55:13.349769 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance
I20260807 10:55:13.351150 3367 external_mini_cluster.cc:1428] Setting key 464eb04092033e03a38cd8f4f3880df1
I20260807 10:55:13.373171 30661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:13.354806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:46183 (local address 127.3.73.254:44115)
0807 10:55:13.355879 (+ 1073us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:13.355904 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:13.358461 (+ 2557us) server_negotiation.cc:407] Connection header received
0807 10:55:13.361820 (+ 3359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:13.361837 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:13.362016 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:13.362228 (+ 212us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:13.364574 (+ 2346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:13.366043 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:13.368638 (+ 2595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:13.369375 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:13.369622 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:13.371555 (+ 1933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:13.371561 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:13.371564 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:13.371585 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:13.371629 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:13.371655 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:13.371829 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:13.372256 (+ 427us) server_negotiation.cc:299] Negotiation successful
0807 10:55:13.372432 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":795,"thread_start_us":512,"threads_started":1}
I20260807 10:55:13.375190 30663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:13.356060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:44115 (local address 127.3.73.195:46183)
0807 10:55:13.358111 (+ 2051us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:13.358232 (+ 121us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:13.361243 (+ 3011us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:13.362749 (+ 1506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:13.362768 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:13.363002 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:55:13.364245 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:13.364287 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:13.366310 (+ 2023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:13.366318 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:13.368386 (+ 2068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:13.368407 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:13.369809 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:13.369830 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:13.370192 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:13.371152 (+ 960us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 10:55:13.371155 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:55:13.371202 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 10:55:13.371308 (+ 106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:55:13.371817 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:55:13.371841 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:55:13.371864 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 10:55:13.372239 (+ 375us) client_negotiation.cc:240] Negotiation successful
0807 10:55:13.373129 (+ 890us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":996,"thread_start_us":647,"threads_started":1}
I20260807 10:55:13.377923 30659 heartbeater.cc:347] Connected to a master server at 127.3.73.254:44115
I20260807 10:55:13.378667 30659 heartbeater.cc:464] Registering TS with master...
I20260807 10:55:13.380102 30659 heartbeater.cc:511] Master 127.3.73.254:44115 requested a full tablet report, sending...
I20260807 10:55:13.382997 29993 ts_manager.cc:194] Registered new tserver with Master: fcb702762962400783361d3faf740d63 (127.3.73.195:36965)
I20260807 10:55:13.384500 29993 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.73.195:46183
I20260807 10:55:13.385053 3367 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 10:55:13.404078 30661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:13.392338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50132 (local address 127.3.73.254:44115)
0807 10:55:13.392687 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:13.392691 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:13.393473 (+ 782us) server_negotiation.cc:407] Connection header received
0807 10:55:13.393814 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:13.393819 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:13.393922 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:13.394118 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:13.395107 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:13.396597 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:13.399242 (+ 2645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:13.399927 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:13.400107 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:13.402862 (+ 2755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:13.402876 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:13.402881 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:13.402910 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:13.402974 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:13.403006 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:13.403144 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:13.403482 (+ 338us) server_negotiation.cc:299] Negotiation successful
0807 10:55:13.403633 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153}
I20260807 10:55:13.411995 29993 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50132:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 10:55:13.429241 29993 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 10:55:13.472878 30680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:13.452323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:36965 (local address 127.0.0.1:42736)
0807 10:55:13.455998 (+ 3675us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:13.456034 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:13.456252 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:13.457793 (+ 1541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:13.457806 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:13.457863 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:55:13.458983 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:13.459011 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:13.463138 (+ 4127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:13.463144 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:13.467266 (+ 4122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:13.467293 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:13.470981 (+ 3688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:13.470999 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:13.471399 (+ 400us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:13.471484 (+ 85us) client_negotiation.cc:769] Sending connection context
0807 10:55:13.471789 (+ 305us) client_negotiation.cc:240] Negotiation successful
0807 10:55:13.472094 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3497,"mutex_wait_us":247,"thread_start_us":1750,"threads_started":1}
I20260807 10:55:13.472970 30676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:13.451856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:42111 (local address 127.0.0.1:52622)
0807 10:55:13.454809 (+ 2953us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:13.454884 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:13.455288 (+ 404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:13.459011 (+ 3723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:13.459028 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:13.459100 (+ 72us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:55:13.460101 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:13.460129 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:13.463351 (+ 3222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:13.463419 (+ 68us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:13.466611 (+ 3192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:13.466647 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:13.470513 (+ 3866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:13.470521 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:13.471014 (+ 493us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:13.471150 (+ 136us) client_negotiation.cc:769] Sending connection context
0807 10:55:13.471506 (+ 356us) client_negotiation.cc:240] Negotiation successful
0807 10:55:13.471893 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2516,"mutex_wait_us":214,"spinlock_wait_cycles":24832,"thread_start_us":6382,"threads_started":1}
I20260807 10:55:13.474901 30677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:13.452525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52622 (local address 127.3.73.193:42111)
0807 10:55:13.457292 (+ 4767us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:13.457315 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:13.457363 (+ 48us) server_negotiation.cc:407] Connection header received
0807 10:55:13.457610 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:13.457650 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:13.458058 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:13.458521 (+ 463us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:55:13.460683 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:13.463135 (+ 2452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:13.466939 (+ 3804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:13.469403 (+ 2464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:13.469663 (+ 260us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:13.470243 (+ 580us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:13.473883 (+ 3640us) server_negotiation.cc:299] Negotiation successful
0807 10:55:13.474174 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4455,"thread_start_us":4167,"threads_started":1}
I20260807 10:55:13.475030 30675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:13.454148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:39411 (local address 127.0.0.1:43734)
0807 10:55:13.458793 (+ 4645us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:13.458824 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:13.459252 (+ 428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:13.460923 (+ 1671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:13.460932 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:13.460963 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:55:13.461421 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:13.461433 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:13.464381 (+ 2948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:13.464387 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:13.468259 (+ 3872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:13.468288 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:13.472583 (+ 4295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:13.472603 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:13.473908 (+ 1305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:13.473957 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 10:55:13.474105 (+ 148us) client_negotiation.cc:240] Negotiation successful
0807 10:55:13.474371 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4415,"mutex_wait_us":8,"spinlock_wait_cycles":2304,"thread_start_us":2007,"threads_started":1}
I20260807 10:55:13.475665 30679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:13.452694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42736 (local address 127.3.73.195:36965)
0807 10:55:13.456415 (+ 3721us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:13.456449 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:13.456531 (+ 82us) server_negotiation.cc:407] Connection header received
0807 10:55:13.456773 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:13.456798 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:13.457343 (+ 545us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:13.459413 (+ 2070us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:55:13.460101 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:13.462848 (+ 2747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:13.467622 (+ 4774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:13.470230 (+ 2608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:13.470522 (+ 292us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:13.471061 (+ 539us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:13.473441 (+ 2380us) server_negotiation.cc:299] Negotiation successful
0807 10:55:13.473886 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3433,"thread_start_us":3140,"threads_started":1}
I20260807 10:55:13.475749 30681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:13.455289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43734 (local address 127.3.73.194:39411)
0807 10:55:13.457848 (+ 2559us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:13.457878 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:13.459038 (+ 1160us) server_negotiation.cc:407] Connection header received
0807 10:55:13.459672 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:13.459685 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:13.460121 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:13.460488 (+ 367us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:55:13.461707 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:13.464084 (+ 2377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:13.468711 (+ 4627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:13.471783 (+ 3072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:13.472523 (+ 740us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:13.473620 (+ 1097us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:13.474275 (+ 655us) server_negotiation.cc:299] Negotiation successful
0807 10:55:13.474647 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2220,"thread_start_us":1978,"threads_started":1}
I20260807 10:55:13.482043 30172 tablet_service.cc:1467] Processing CreateTablet for tablet 47925709d1954fcabf2873757177613c (DEFAULT_TABLE table=test-table [id=6697d854c7b14953976c5954b74085c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:55:13.483268 30370 tablet_service.cc:1467] Processing CreateTablet for tablet 47925709d1954fcabf2873757177613c (DEFAULT_TABLE table=test-table [id=6697d854c7b14953976c5954b74085c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:55:13.483268 30566 tablet_service.cc:1467] Processing CreateTablet for tablet 47925709d1954fcabf2873757177613c (DEFAULT_TABLE table=test-table [id=6697d854c7b14953976c5954b74085c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:55:13.485751 30172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47925709d1954fcabf2873757177613c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:55:13.485930 30566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47925709d1954fcabf2873757177613c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:55:13.485949 30370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47925709d1954fcabf2873757177613c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:55:13.509130 30682 tablet_bootstrap.cc:492] T 47925709d1954fcabf2873757177613c P 7cf3d5f3af9a4c2387a687b65feb11ff: Bootstrap starting.
I20260807 10:55:13.513615 30683 tablet_bootstrap.cc:492] T 47925709d1954fcabf2873757177613c P 0205eed7f4c24db5963189e4391818af: Bootstrap starting.
I20260807 10:55:13.514879 30684 tablet_bootstrap.cc:492] T 47925709d1954fcabf2873757177613c P fcb702762962400783361d3faf740d63: Bootstrap starting.
I20260807 10:55:13.518546 30682 tablet_bootstrap.cc:654] T 47925709d1954fcabf2873757177613c P 7cf3d5f3af9a4c2387a687b65feb11ff: Neither blocks nor log segments found. Creating new log.
I20260807 10:55:13.520208 30684 tablet_bootstrap.cc:654] T 47925709d1954fcabf2873757177613c P fcb702762962400783361d3faf740d63: Neither blocks nor log segments found. Creating new log.
I20260807 10:55:13.521065 30682 log.cc:824] T 47925709d1954fcabf2873757177613c P 7cf3d5f3af9a4c2387a687b65feb11ff: Log is configured to *not* fsync() on all Append() calls
I20260807 10:55:13.521589 30683 tablet_bootstrap.cc:654] T 47925709d1954fcabf2873757177613c P 0205eed7f4c24db5963189e4391818af: Neither blocks nor log segments found. Creating new log.
I20260807 10:55:13.521942 30684 log.cc:824] T 47925709d1954fcabf2873757177613c P fcb702762962400783361d3faf740d63: Log is configured to *not* fsync() on all Append() calls
I20260807 10:55:13.523990 30683 log.cc:824] T 47925709d1954fcabf2873757177613c P 0205eed7f4c24db5963189e4391818af: Log is configured to *not* fsync() on all Append() calls
I20260807 10:55:13.527058 30682 tablet_bootstrap.cc:492] T 47925709d1954fcabf2873757177613c P 7cf3d5f3af9a4c2387a687b65feb11ff: No bootstrap required, opened a new log
I20260807 10:55:13.527573 30682 ts_tablet_manager.cc:1397] T 47925709d1954fcabf2873757177613c P 7cf3d5f3af9a4c2387a687b65feb11ff: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.003s
I20260807 10:55:13.528227 30684 tablet_bootstrap.cc:492] T 47925709d1954fcabf2873757177613c P fcb702762962400783361d3faf740d63: No bootstrap required, opened a new log
I20260807 10:55:13.528959 30684 ts_tablet_manager.cc:1397] T 47925709d1954fcabf2873757177613c P fcb702762962400783361d3faf740d63: Time spent bootstrapping tablet: real 0.014s user 0.012s sys 0.000s
I20260807 10:55:13.530134 30683 tablet_bootstrap.cc:492] T 47925709d1954fcabf2873757177613c P 0205eed7f4c24db5963189e4391818af: No bootstrap required, opened a new log
I20260807 10:55:13.530893 30683 ts_tablet_manager.cc:1397] T 47925709d1954fcabf2873757177613c P 0205eed7f4c24db5963189e4391818af: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.005s
I20260807 10:55:13.544270 30683 raft_consensus.cc:348] T 47925709d1954fcabf2873757177613c P 0205eed7f4c24db5963189e4391818af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cf3d5f3af9a4c2387a687b65feb11ff" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 42111 } } peers { permanent_uuid: "fcb702762962400783361d3faf740d63" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36965 } } peers { permanent_uuid: "0205eed7f4c24db5963189e4391818af" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 39411 } }
I20260807 10:55:13.545426 30683 raft_consensus.cc:374] T 47925709d1954fcabf2873757177613c P 0205eed7f4c24db5963189e4391818af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:55:13.545605 30683 raft_consensus.cc:729] T 47925709d1954fcabf2873757177613c P 0205eed7f4c24db5963189e4391818af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0205eed7f4c24db5963189e4391818af, State: Initialized, Role: FOLLOWER
I20260807 10:55:13.547228 30683 consensus_queue.cc:260] T 47925709d1954fcabf2873757177613c P 0205eed7f4c24db5963189e4391818af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cf3d5f3af9a4c2387a687b65feb11ff" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 42111 } } peers { permanent_uuid: "fcb702762962400783361d3faf740d63" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36965 } } peers { permanent_uuid: "0205eed7f4c24db5963189e4391818af" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 39411 } }
I20260807 10:55:13.548463 30682 raft_consensus.cc:348] T 47925709d1954fcabf2873757177613c P 7cf3d5f3af9a4c2387a687b65feb11ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cf3d5f3af9a4c2387a687b65feb11ff" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 42111 } } peers { permanent_uuid: "fcb702762962400783361d3faf740d63" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36965 } } peers { permanent_uuid: "0205eed7f4c24db5963189e4391818af" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 39411 } }
I20260807 10:55:13.548472 30684 raft_consensus.cc:348] T 47925709d1954fcabf2873757177613c P fcb702762962400783361d3faf740d63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cf3d5f3af9a4c2387a687b65feb11ff" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 42111 } } peers { permanent_uuid: "fcb702762962400783361d3faf740d63" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36965 } } peers { permanent_uuid: "0205eed7f4c24db5963189e4391818af" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 39411 } }
I20260807 10:55:13.549464 30684 raft_consensus.cc:374] T 47925709d1954fcabf2873757177613c P fcb702762962400783361d3faf740d63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:55:13.549506 30682 raft_consensus.cc:374] T 47925709d1954fcabf2873757177613c P 7cf3d5f3af9a4c2387a687b65feb11ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:55:13.549665 30682 raft_consensus.cc:729] T 47925709d1954fcabf2873757177613c P 7cf3d5f3af9a4c2387a687b65feb11ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cf3d5f3af9a4c2387a687b65feb11ff, State: Initialized, Role: FOLLOWER
I20260807 10:55:13.549757 30684 raft_consensus.cc:729] T 47925709d1954fcabf2873757177613c P fcb702762962400783361d3faf740d63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcb702762962400783361d3faf740d63, State: Initialized, Role: FOLLOWER
I20260807 10:55:13.550299 30464 heartbeater.cc:503] Master 127.3.73.254:44115 was elected leader, sending a full tablet report...
I20260807 10:55:13.551034 30684 consensus_queue.cc:260] T 47925709d1954fcabf2873757177613c P fcb702762962400783361d3faf740d63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cf3d5f3af9a4c2387a687b65feb11ff" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 42111 } } peers { permanent_uuid: "fcb702762962400783361d3faf740d63" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36965 } } peers { permanent_uuid: "0205eed7f4c24db5963189e4391818af" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 39411 } }
I20260807 10:55:13.551123 30682 consensus_queue.cc:260] T 47925709d1954fcabf2873757177613c P 7cf3d5f3af9a4c2387a687b65feb11ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cf3d5f3af9a4c2387a687b65feb11ff" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 42111 } } peers { permanent_uuid: "fcb702762962400783361d3faf740d63" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36965 } } peers { permanent_uuid: "0205eed7f4c24db5963189e4391818af" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 39411 } }
I20260807 10:55:13.551635 30683 ts_tablet_manager.cc:1428] T 47925709d1954fcabf2873757177613c P 0205eed7f4c24db5963189e4391818af: Time spent starting tablet: real 0.021s user 0.016s sys 0.003s
I20260807 10:55:13.554440 30684 ts_tablet_manager.cc:1428] T 47925709d1954fcabf2873757177613c P fcb702762962400783361d3faf740d63: Time spent starting tablet: real 0.025s user 0.023s sys 0.000s
I20260807 10:55:13.555205 30682 ts_tablet_manager.cc:1428] T 47925709d1954fcabf2873757177613c P 7cf3d5f3af9a4c2387a687b65feb11ff: Time spent starting tablet: real 0.027s user 0.023s sys 0.005s
I20260807 10:55:13.555526 30659 heartbeater.cc:503] Master 127.3.73.254:44115 was elected leader, sending a full tablet report...
W20260807 10:55:13.602877 30660 tablet.cc:2369] T 47925709d1954fcabf2873757177613c P fcb702762962400783361d3faf740d63: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 10:55:13.612660 30277 tablet.cc:2369] T 47925709d1954fcabf2873757177613c P 7cf3d5f3af9a4c2387a687b65feb11ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 10:55:13.629245 30465 tablet.cc:2369] T 47925709d1954fcabf2873757177613c P 0205eed7f4c24db5963189e4391818af: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:55:13.912853 30690 raft_consensus.cc:482] T 47925709d1954fcabf2873757177613c P 0205eed7f4c24db5963189e4391818af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 10:55:13.913458 30690 raft_consensus.cc:504] T 47925709d1954fcabf2873757177613c P 0205eed7f4c24db5963189e4391818af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cf3d5f3af9a4c2387a687b65feb11ff" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 42111 } } peers { permanent_uuid: "fcb702762962400783361d3faf740d63" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36965 } } peers { permanent_uuid: "0205eed7f4c24db5963189e4391818af" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 39411 } }
I20260807 10:55:13.916301 30690 leader_election.cc:288] T 47925709d1954fcabf2873757177613c P 0205eed7f4c24db5963189e4391818af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cf3d5f3af9a4c2387a687b65feb11ff (127.3.73.193:42111), fcb702762962400783361d3faf740d63 (127.3.73.195:36965)
I20260807 10:55:13.928299 30677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:13.917121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:45755 (local address 127.3.73.193:42111)
0807 10:55:13.917556 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:13.917566 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:13.918938 (+ 1372us) server_negotiation.cc:407] Connection header received
0807 10:55:13.919198 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:13.919204 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:13.919306 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:13.919508 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:55:13.920548 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:13.921636 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:13.924581 (+ 2945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:13.926056 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:13.926535 (+ 479us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:13.926720 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:13.927429 (+ 709us) server_negotiation.cc:299] Negotiation successful
0807 10:55:13.927722 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
I20260807 10:55:13.928347 30679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:13.917311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:45205 (local address 127.3.73.195:36965)
0807 10:55:13.917694 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:13.917717 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:13.918668 (+ 951us) server_negotiation.cc:407] Connection header received
0807 10:55:13.919098 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:13.919103 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:13.919197 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:13.919376 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:55:13.920303 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:13.921425 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:13.924541 (+ 3116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:13.925971 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:13.926447 (+ 476us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:13.926621 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:13.927401 (+ 780us) server_negotiation.cc:299] Negotiation successful
0807 10:55:13.927647 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
I20260807 10:55:13.928673 30695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:13.917035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:36965 (local address 127.3.73.194:45205)
0807 10:55:13.918469 (+ 1434us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:13.918508 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:13.918786 (+ 278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:13.919409 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:13.919415 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:13.919461 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:55:13.920057 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:13.920071 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:13.921665 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:13.921684 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:13.924176 (+ 2492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:13.924210 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:13.926437 (+ 2227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:13.926457 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:13.926948 (+ 491us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:13.927073 (+ 125us) client_negotiation.cc:769] Sending connection context
0807 10:55:13.927360 (+ 287us) client_negotiation.cc:240] Negotiation successful
0807 10:55:13.927655 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1247,"mutex_wait_us":164,"thread_start_us":846,"threads_started":1}
I20260807 10:55:13.929242 30696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:13.917044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:42111 (local address 127.3.73.194:45755)
0807 10:55:13.918769 (+ 1725us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:13.918786 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:13.918966 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:13.919515 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:13.919520 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:13.919564 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:55:13.920285 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:13.920306 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:13.921886 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:13.921901 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:13.924193 (+ 2292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:13.924225 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:13.926529 (+ 2304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:13.926538 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:13.927095 (+ 557us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:13.927158 (+ 63us) client_negotiation.cc:769] Sending connection context
0807 10:55:13.927377 (+ 219us) client_negotiation.cc:240] Negotiation successful
0807 10:55:13.927678 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1554,"mutex_wait_us":156,"thread_start_us":1184,"threads_started":1}
I20260807 10:55:13.930946 30593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47925709d1954fcabf2873757177613c" candidate_uuid: "0205eed7f4c24db5963189e4391818af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb702762962400783361d3faf740d63" is_pre_election: true
I20260807 10:55:13.931986 30593 raft_consensus.cc:2454] T 47925709d1954fcabf2873757177613c P fcb702762962400783361d3faf740d63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0205eed7f4c24db5963189e4391818af in term 0.
I20260807 10:55:13.931577 30199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47925709d1954fcabf2873757177613c" candidate_uuid: "0205eed7f4c24db5963189e4391818af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cf3d5f3af9a4c2387a687b65feb11ff" is_pre_election: true
I20260807 10:55:13.932837 30199 raft_consensus.cc:2454] T 47925709d1954fcabf2873757177613c P 7cf3d5f3af9a4c2387a687b65feb11ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0205eed7f4c24db5963189e4391818af in term 0.
I20260807 10:55:13.933561 30305 leader_election.cc:302] T 47925709d1954fcabf2873757177613c P 0205eed7f4c24db5963189e4391818af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0205eed7f4c24db5963189e4391818af, fcb702762962400783361d3faf740d63; no voters:
I20260807 10:55:13.934535 30690 raft_consensus.cc:2781] T 47925709d1954fcabf2873757177613c P 0205eed7f4c24db5963189e4391818af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 10:55:13.934864 30690 raft_consensus.cc:482] T 47925709d1954fcabf2873757177613c P 0205eed7f4c24db5963189e4391818af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 10:55:13.935072 30690 raft_consensus.cc:3037] T 47925709d1954fcabf2873757177613c P 0205eed7f4c24db5963189e4391818af [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:55:13.939669 30690 raft_consensus.cc:504] T 47925709d1954fcabf2873757177613c P 0205eed7f4c24db5963189e4391818af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cf3d5f3af9a4c2387a687b65feb11ff" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 42111 } } peers { permanent_uuid: "fcb702762962400783361d3faf740d63" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36965 } } peers { permanent_uuid: "0205eed7f4c24db5963189e4391818af" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 39411 } }
I20260807 10:55:13.941080 30690 leader_election.cc:288] T 47925709d1954fcabf2873757177613c P 0205eed7f4c24db5963189e4391818af [CANDIDATE]: Term 1 election: Requested vote from peers 7cf3d5f3af9a4c2387a687b65feb11ff (127.3.73.193:42111), fcb702762962400783361d3faf740d63 (127.3.73.195:36965)
I20260807 10:55:13.942037 30199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47925709d1954fcabf2873757177613c" candidate_uuid: "0205eed7f4c24db5963189e4391818af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cf3d5f3af9a4c2387a687b65feb11ff"
I20260807 10:55:13.942153 30593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47925709d1954fcabf2873757177613c" candidate_uuid: "0205eed7f4c24db5963189e4391818af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcb702762962400783361d3faf740d63"
I20260807 10:55:13.942417 30199 raft_consensus.cc:3037] T 47925709d1954fcabf2873757177613c P 7cf3d5f3af9a4c2387a687b65feb11ff [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:55:13.942617 30593 raft_consensus.cc:3037] T 47925709d1954fcabf2873757177613c P fcb702762962400783361d3faf740d63 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:55:13.946416 30199 raft_consensus.cc:2454] T 47925709d1954fcabf2873757177613c P 7cf3d5f3af9a4c2387a687b65feb11ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0205eed7f4c24db5963189e4391818af in term 1.
I20260807 10:55:13.946630 30593 raft_consensus.cc:2454] T 47925709d1954fcabf2873757177613c P fcb702762962400783361d3faf740d63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0205eed7f4c24db5963189e4391818af in term 1.
I20260807 10:55:13.947365 30303 leader_election.cc:302] T 47925709d1954fcabf2873757177613c P 0205eed7f4c24db5963189e4391818af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0205eed7f4c24db5963189e4391818af, 7cf3d5f3af9a4c2387a687b65feb11ff; no voters:
I20260807 10:55:13.947932 30690 raft_consensus.cc:2781] T 47925709d1954fcabf2873757177613c P 0205eed7f4c24db5963189e4391818af [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:55:13.948695 30690 raft_consensus.cc:686] T 47925709d1954fcabf2873757177613c P 0205eed7f4c24db5963189e4391818af [term 1 LEADER]: Becoming Leader. State: Replica: 0205eed7f4c24db5963189e4391818af, State: Running, Role: LEADER
I20260807 10:55:13.949724 30690 consensus_queue.cc:237] T 47925709d1954fcabf2873757177613c P 0205eed7f4c24db5963189e4391818af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cf3d5f3af9a4c2387a687b65feb11ff" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 42111 } } peers { permanent_uuid: "fcb702762962400783361d3faf740d63" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36965 } } peers { permanent_uuid: "0205eed7f4c24db5963189e4391818af" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 39411 } }
I20260807 10:55:13.959345 29994 catalog_manager.cc:5495] T 47925709d1954fcabf2873757177613c P 0205eed7f4c24db5963189e4391818af reported cstate change: term changed from 0 to 1, leader changed from <none> to 0205eed7f4c24db5963189e4391818af (127.3.73.194). New cstate: current_term: 1 leader_uuid: "0205eed7f4c24db5963189e4391818af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cf3d5f3af9a4c2387a687b65feb11ff" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 42111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fcb702762962400783361d3faf740d63" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0205eed7f4c24db5963189e4391818af" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 39411 } health_report { overall_health: HEALTHY } } }
I20260807 10:55:14.067113 30706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:14.056392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43740 (local address 127.3.73.194:39411)
0807 10:55:14.057310 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:14.057318 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:14.057667 (+ 349us) server_negotiation.cc:407] Connection header received
0807 10:55:14.057968 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:14.057973 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:14.058162 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:14.058473 (+ 311us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:55:14.059138 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:14.060588 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:14.063172 (+ 2584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:14.063947 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:14.064092 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:14.065321 (+ 1229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:55:14.066283 (+ 962us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:55:14.066464 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:14.066665 (+ 201us) server_negotiation.cc:299] Negotiation successful
0807 10:55:14.066770 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":705,"thread_start_us":489,"threads_started":1}
I20260807 10:55:14.081193 30199 raft_consensus.cc:1261] T 47925709d1954fcabf2873757177613c P 7cf3d5f3af9a4c2387a687b65feb11ff [term 1 FOLLOWER]: Refusing update from remote peer 0205eed7f4c24db5963189e4391818af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:55:14.081328 30593 raft_consensus.cc:1261] T 47925709d1954fcabf2873757177613c P fcb702762962400783361d3faf740d63 [term 1 FOLLOWER]: Refusing update from remote peer 0205eed7f4c24db5963189e4391818af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:55:14.083055 30697 consensus_queue.cc:1035] T 47925709d1954fcabf2873757177613c P 0205eed7f4c24db5963189e4391818af [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcb702762962400783361d3faf740d63" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:55:14.083741 30690 consensus_queue.cc:1035] T 47925709d1954fcabf2873757177613c P 0205eed7f4c24db5963189e4391818af [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cf3d5f3af9a4c2387a687b65feb11ff" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 42111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:55:14.105439 30708 mvcc.cc:205] Tried to move back new op lower bound from 7315866067257810944 to 7315866066745483264. Current Snapshot: MvccSnapshot[applied={T|T < 7315866067257810944}]
I20260807 10:55:14.124935 30712 mvcc.cc:205] Tried to move back new op lower bound from 7315866067257810944 to 7315866066745483264. Current Snapshot: MvccSnapshot[applied={T|T < 7315866067257810944}]
I20260807 10:55:14.143898 29994 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:55:14.144411 29994 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:55:14.152025 29994 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e13c5e492fa746aab2cee50be5b626b8: Sending DeleteTablet for 3 replicas of tablet 47925709d1954fcabf2873757177613c
I20260807 10:55:14.154826 30172 tablet_service.cc:1514] Processing DeleteTablet for tablet 47925709d1954fcabf2873757177613c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:55:14 UTC) from {username='slave'} at 127.0.0.1:52622
I20260807 10:55:14.154824 30566 tablet_service.cc:1514] Processing DeleteTablet for tablet 47925709d1954fcabf2873757177613c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:55:14 UTC) from {username='slave'} at 127.0.0.1:42736
I20260807 10:55:14.155066 30370 tablet_service.cc:1514] Processing DeleteTablet for tablet 47925709d1954fcabf2873757177613c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:55:14 UTC) from {username='slave'} at 127.0.0.1:43734
I20260807 10:55:14.161536 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 30089
I20260807 10:55:14.170075 30724 tablet_replica.cc:330] T 47925709d1954fcabf2873757177613c P 7cf3d5f3af9a4c2387a687b65feb11ff: stopping tablet replica
I20260807 10:55:14.170866 30724 raft_consensus.cc:2229] T 47925709d1954fcabf2873757177613c P 7cf3d5f3af9a4c2387a687b65feb11ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:55:14.171990 30724 raft_consensus.cc:2258] T 47925709d1954fcabf2873757177613c P 7cf3d5f3af9a4c2387a687b65feb11ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:55:14.177896 30727 tablet_replica.cc:330] T 47925709d1954fcabf2873757177613c P 0205eed7f4c24db5963189e4391818af: stopping tablet replica
I20260807 10:55:14.180572 30724 ts_tablet_manager.cc:1905] T 47925709d1954fcabf2873757177613c P 7cf3d5f3af9a4c2387a687b65feb11ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:55:14.181530 30727 raft_consensus.cc:2229] T 47925709d1954fcabf2873757177613c P 0205eed7f4c24db5963189e4391818af [term 1 LEADER]: Raft consensus shutting down.
I20260807 10:55:14.188832 30727 raft_consensus.cc:2258] T 47925709d1954fcabf2873757177613c P 0205eed7f4c24db5963189e4391818af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:55:14.189806 30726 tablet_replica.cc:330] T 47925709d1954fcabf2873757177613c P fcb702762962400783361d3faf740d63: stopping tablet replica
I20260807 10:55:14.190726 30726 raft_consensus.cc:2229] T 47925709d1954fcabf2873757177613c P fcb702762962400783361d3faf740d63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:55:14.191691 30726 raft_consensus.cc:2258] T 47925709d1954fcabf2873757177613c P fcb702762962400783361d3faf740d63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:55:14.195209 30724 ts_tablet_manager.cc:1918] T 47925709d1954fcabf2873757177613c P 7cf3d5f3af9a4c2387a687b65feb11ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:55:14.195576 30724 log.cc:1196] T 47925709d1954fcabf2873757177613c P 7cf3d5f3af9a4c2387a687b65feb11ff: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-0/wal/wals/47925709d1954fcabf2873757177613c
I20260807 10:55:14.196273 30726 ts_tablet_manager.cc:1905] T 47925709d1954fcabf2873757177613c P fcb702762962400783361d3faf740d63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:55:14.197057 30724 ts_tablet_manager.cc:1939] T 47925709d1954fcabf2873757177613c P 7cf3d5f3af9a4c2387a687b65feb11ff: Deleting consensus metadata
I20260807 10:55:14.197978 30727 ts_tablet_manager.cc:1905] T 47925709d1954fcabf2873757177613c P 0205eed7f4c24db5963189e4391818af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:55:14.200850 29974 catalog_manager.cc:4841] TS 7cf3d5f3af9a4c2387a687b65feb11ff (127.3.73.193:42111): tablet 47925709d1954fcabf2873757177613c (table test-table [id=6697d854c7b14953976c5954b74085c0]) successfully deleted
I20260807 10:55:14.205899 30677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:14.164734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52638 (local address 127.3.73.193:42111)
0807 10:55:14.165234 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:14.165257 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:14.176420 (+ 11163us) server_negotiation.cc:407] Connection header received
0807 10:55:14.178790 (+ 2370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:14.178799 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:14.179010 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:14.179191 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:14.182034 (+ 2843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:14.184035 (+ 2001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:14.192193 (+ 8158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:14.193674 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:14.193923 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:14.202326 (+ 8403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:14.202353 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:14.202386 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:14.203972 (+ 1586us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:14.204088 (+ 116us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:14.204125 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:14.204248 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:14.204848 (+ 600us) server_negotiation.cc:299] Negotiation successful
0807 10:55:14.205165 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219}
I20260807 10:55:14.208460 30264 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:55:14.211318 30726 ts_tablet_manager.cc:1918] T 47925709d1954fcabf2873757177613c P fcb702762962400783361d3faf740d63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:55:14.211727 30726 log.cc:1196] T 47925709d1954fcabf2873757177613c P fcb702762962400783361d3faf740d63: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-2/wal/wals/47925709d1954fcabf2873757177613c
I20260807 10:55:14.212853 30726 ts_tablet_manager.cc:1939] T 47925709d1954fcabf2873757177613c P fcb702762962400783361d3faf740d63: Deleting consensus metadata
I20260807 10:55:14.214789 30727 ts_tablet_manager.cc:1918] T 47925709d1954fcabf2873757177613c P 0205eed7f4c24db5963189e4391818af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:55:14.215212 30727 log.cc:1196] T 47925709d1954fcabf2873757177613c P 0205eed7f4c24db5963189e4391818af: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786099767749326-3367-0/minicluster-data/ts-1/wal/wals/47925709d1954fcabf2873757177613c
I20260807 10:55:14.216465 30727 ts_tablet_manager.cc:1939] T 47925709d1954fcabf2873757177613c P 0205eed7f4c24db5963189e4391818af: Deleting consensus metadata
I20260807 10:55:14.216958 29978 catalog_manager.cc:4841] TS fcb702762962400783361d3faf740d63 (127.3.73.195:36965): tablet 47925709d1954fcabf2873757177613c (table test-table [id=6697d854c7b14953976c5954b74085c0]) successfully deleted
I20260807 10:55:14.220968 29980 catalog_manager.cc:4841] TS 0205eed7f4c24db5963189e4391818af (127.3.73.194:39411): tablet 47925709d1954fcabf2873757177613c (table test-table [id=6697d854c7b14953976c5954b74085c0]) successfully deleted
I20260807 10:55:14.331936 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 30089
I20260807 10:55:14.369436 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 30283
I20260807 10:55:14.378554 30706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:14.370551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43750 (local address 127.3.73.194:39411)
0807 10:55:14.370788 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:14.370796 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:14.370820 (+ 24us) server_negotiation.cc:407] Connection header received
0807 10:55:14.370900 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:14.370906 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:14.371009 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:14.371127 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:14.371678 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:14.372554 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:14.374626 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:14.375125 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:14.375239 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:14.376662 (+ 1423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:14.376672 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:14.376689 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:14.377508 (+ 819us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:14.377576 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:14.377601 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:14.377699 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:14.378003 (+ 304us) server_negotiation.cc:299] Negotiation successful
0807 10:55:14.378111 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260807 10:55:14.379456 30457 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:55:14.497948 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 30283
I20260807 10:55:14.527107 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 30469
I20260807 10:55:14.535199 30735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:14.528080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42738 (local address 127.3.73.195:36965)
0807 10:55:14.528653 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:14.528658 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:14.528676 (+ 18us) server_negotiation.cc:407] Connection header received
0807 10:55:14.528796 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:14.528800 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:14.528892 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:14.528992 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:14.529440 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:14.530200 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:14.531601 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:14.532143 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:14.532268 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:14.533676 (+ 1408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:14.533684 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:14.533700 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:14.534358 (+ 658us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:14.534422 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:14.534444 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:14.534546 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:14.534789 (+ 243us) server_negotiation.cc:299] Negotiation successful
0807 10:55:14.534900 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":346,"threads_started":1}
I20260807 10:55:14.536031 30652 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:55:14.629048 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 30469
I20260807 10:55:14.654284 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 29958
I20260807 10:55:14.655561 30036 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:55:14.774415 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 29958
2026-08-07T10:55:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3986 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:55:14 dist-test-slave-x5fp krb5kdc[30744](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:55:14 dist-test-slave-x5fp krb5kdc[30744](info): set up 2 sockets
Aug 07 10:55:14 dist-test-slave-x5fp krb5kdc[30744](info): commencing operation
krb5kdc: starting...
W20260807 10:55:16.183434 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.347s user 0.000s sys 0.060s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:55:16 dist-test-slave-x5fp krb5kdc[30744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:55:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:55:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.73.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
I20260807 10:55:16.573359 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:37719
--webserver_interface=127.3.73.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:33907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:37719
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:55:16.952114 30769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:55:16.952478 30769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:55:16.952580 30769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:55:16.960294 30769 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:55:16.960429 30769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:55:16.960479 30769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:55:16.960531 30769 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:55:16.960577 30769 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:55:16.971570 30769 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:33907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:37719
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:37719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:55:16.973075 30769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:55:16.974529 30769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:55:16.987560 30776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:16.987722 30778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:16.988788 30769 server_base.cc:1034] running on GCE node
W20260807 10:55:16.988839 30780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:16.989825 30769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:55:16.991601 30769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:55:16.992870 30769 hybrid_clock.cc:648] HybridClock initialized: now 1786100116992818 us; error 83 us; skew 500 ppm
I20260807 10:55:16.993278 30769 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:55:16 dist-test-slave-x5fp krb5kdc[30744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100116, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:55:17.004245 30769 init.cc:377] Logged in from keytab as kudu/127.3.73.254@KRBTEST.COM (short username kudu)
I20260807 10:55:17.006383 30769 webserver.cc:460] Webserver started at http://127.3.73.254:46863/ using document root <none> and password file <none>
I20260807 10:55:17.007266 30769 fs_manager.cc:359] Metadata directory not provided
I20260807 10:55:17.007463 30769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:55:17.007855 30769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:55:17.011533 30769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "fc852db596474e50b9ba0aa802770eb8"
format_stamp: "Formatted at 2026-08-07 10:55:17 on dist-test-slave-x5fp"
server_key: "857f76665c02c9277628cfbb3bc04cfd"
server_key_iv: "8bd24cd06076989ff75d2f1bfda9ce08"
server_key_version: "encryptionkey@0"
I20260807 10:55:17.012626 30769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "fc852db596474e50b9ba0aa802770eb8"
format_stamp: "Formatted at 2026-08-07 10:55:17 on dist-test-slave-x5fp"
server_key: "857f76665c02c9277628cfbb3bc04cfd"
server_key_iv: "8bd24cd06076989ff75d2f1bfda9ce08"
server_key_version: "encryptionkey@0"
I20260807 10:55:17.018903 30769 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260807 10:55:17.023016 30792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:17.024606 30769 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 10:55:17.024855 30769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "fc852db596474e50b9ba0aa802770eb8"
format_stamp: "Formatted at 2026-08-07 10:55:17 on dist-test-slave-x5fp"
server_key: "857f76665c02c9277628cfbb3bc04cfd"
server_key_iv: "8bd24cd06076989ff75d2f1bfda9ce08"
server_key_version: "encryptionkey@0"
I20260807 10:55:17.025094 30769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:55:17.053949 30769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:55:17.058890 30769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:55:17.059239 30769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:55:17.079622 30769 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:37719
I20260807 10:55:17.079679 30881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:37719 every 8 connection(s)
I20260807 10:55:17.081216 30769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:55:17.082722 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 30769
I20260807 10:55:17.082955 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:55:17.083606 3367 external_mini_cluster.cc:1428] Setting key af555c4c7628e30d5c02e59111ea66d7
I20260807 10:55:17.086866 30883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 10:55:17 dist-test-slave-x5fp krb5kdc[30744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:55:17.103073 30883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc852db596474e50b9ba0aa802770eb8: Bootstrap starting.
I20260807 10:55:17.106837 30883 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc852db596474e50b9ba0aa802770eb8: Neither blocks nor log segments found. Creating new log.
I20260807 10:55:17.107977 30883 log.cc:824] T 00000000000000000000000000000000 P fc852db596474e50b9ba0aa802770eb8: Log is configured to *not* fsync() on all Append() calls
I20260807 10:55:17.111568 30883 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc852db596474e50b9ba0aa802770eb8: No bootstrap required, opened a new log
I20260807 10:55:17.111675 30886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:17.086016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52114 (local address 127.3.73.254:37719)
0807 10:55:17.087130 (+ 1114us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:17.087144 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:17.087191 (+ 47us) server_negotiation.cc:407] Connection header received
0807 10:55:17.089111 (+ 1920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:17.089138 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:17.089567 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:17.089957 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:17.092335 (+ 2378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:17.094020 (+ 1685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:17.096415 (+ 2395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:17.097333 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:17.097495 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:17.103059 (+ 5564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:17.103069 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:17.103086 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:55:17.103133 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:17.106622 (+ 3489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:17.107586 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:17.107594 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:17.107601 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:17.107801 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:17.108338 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:17.108343 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:17.108345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:17.109243 (+ 898us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:17.109603 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:17.110118 (+ 515us) server_negotiation.cc:299] Negotiation successful
0807 10:55:17.110562 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":843,"thread_start_us":561,"threads_started":1}
I20260807 10:55:17.119788 30883 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc852db596474e50b9ba0aa802770eb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc852db596474e50b9ba0aa802770eb8" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 37719 } }
I20260807 10:55:17.120282 30883 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc852db596474e50b9ba0aa802770eb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:55:17.120411 30883 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc852db596474e50b9ba0aa802770eb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc852db596474e50b9ba0aa802770eb8, State: Initialized, Role: FOLLOWER
I20260807 10:55:17.121101 30883 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc852db596474e50b9ba0aa802770eb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc852db596474e50b9ba0aa802770eb8" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 37719 } }
I20260807 10:55:17.121393 30883 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc852db596474e50b9ba0aa802770eb8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:55:17.121563 30883 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc852db596474e50b9ba0aa802770eb8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:55:17.121788 30883 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc852db596474e50b9ba0aa802770eb8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:55:17.124150 30883 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc852db596474e50b9ba0aa802770eb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc852db596474e50b9ba0aa802770eb8" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 37719 } }
I20260807 10:55:17.124847 30883 leader_election.cc:302] T 00000000000000000000000000000000 P fc852db596474e50b9ba0aa802770eb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc852db596474e50b9ba0aa802770eb8; no voters:
I20260807 10:55:17.125633 30883 leader_election.cc:288] T 00000000000000000000000000000000 P fc852db596474e50b9ba0aa802770eb8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:55:17.125973 30888 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc852db596474e50b9ba0aa802770eb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:55:17.127477 30888 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc852db596474e50b9ba0aa802770eb8 [term 1 LEADER]: Becoming Leader. State: Replica: fc852db596474e50b9ba0aa802770eb8, State: Running, Role: LEADER
I20260807 10:55:17.128376 30883 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc852db596474e50b9ba0aa802770eb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:55:17.128342 30888 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc852db596474e50b9ba0aa802770eb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc852db596474e50b9ba0aa802770eb8" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 37719 } }
I20260807 10:55:17.133437 30890 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc852db596474e50b9ba0aa802770eb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc852db596474e50b9ba0aa802770eb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc852db596474e50b9ba0aa802770eb8" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 37719 } } }
I20260807 10:55:17.134011 30890 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc852db596474e50b9ba0aa802770eb8 [sys.catalog]: This master's current role is: LEADER
I20260807 10:55:17.135077 30899 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:55:17.134745 30891 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc852db596474e50b9ba0aa802770eb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc852db596474e50b9ba0aa802770eb8. Latest consensus state: current_term: 1 leader_uuid: "fc852db596474e50b9ba0aa802770eb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc852db596474e50b9ba0aa802770eb8" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 37719 } } }
I20260807 10:55:17.135241 30891 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc852db596474e50b9ba0aa802770eb8 [sys.catalog]: This master's current role is: LEADER
I20260807 10:55:17.141868 30899 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:55:17.154192 30899 catalog_manager.cc:1347] Generated new cluster ID: 50d53107ddb446d681f72d0d0a64072f
I20260807 10:55:17.154423 30899 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:55:17.198421 30899 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:55:17.199882 30899 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:55:17.224776 30899 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc852db596474e50b9ba0aa802770eb8: Generated new TSK 0
I20260807 10:55:17.226030 30899 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.73.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab.
Entry for principal kudu/127.3.73.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab.
I20260807 10:55:17.333493 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.193:0
--local_ip_for_outbound_sockets=127.3.73.193
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:37719
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:33907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:55:17.723680 30916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:55:17.724026 30916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:55:17.724205 30916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:55:17.732349 30916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:55:17.732517 30916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:55:17.732638 30916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.193
I20260807 10:55:17.743585 30916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:33907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:37719
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.193
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:55:17.745040 30916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:55:17.746441 30916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:55:17.761304 30922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:17.761361 30927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:17.761309 30923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:17.762251 30916 server_base.cc:1034] running on GCE node
I20260807 10:55:17.762979 30916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:55:17.764115 30916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:55:17.765377 30916 hybrid_clock.cc:648] HybridClock initialized: now 1786100117765319 us; error 49 us; skew 500 ppm
I20260807 10:55:17.765767 30916 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:55:17 dist-test-slave-x5fp krb5kdc[30744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100117, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:55:17.769634 30916 init.cc:377] Logged in from keytab as kudu/127.3.73.193@KRBTEST.COM (short username kudu)
I20260807 10:55:17.771543 30916 webserver.cc:460] Webserver started at http://127.3.73.193:35697/ using document root <none> and password file <none>
I20260807 10:55:17.772316 30916 fs_manager.cc:359] Metadata directory not provided
I20260807 10:55:17.772480 30916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:55:17.772894 30916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:55:17.776211 30916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-0/data/instance:
uuid: "5fe5eee0679745b4863f20034ba77feb"
format_stamp: "Formatted at 2026-08-07 10:55:17 on dist-test-slave-x5fp"
server_key: "24de6befd19b38c72e6b00a1f660e40a"
server_key_iv: "68ce13e0e513d3d150ead8b376346970"
server_key_version: "encryptionkey@0"
I20260807 10:55:17.777179 30916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance:
uuid: "5fe5eee0679745b4863f20034ba77feb"
format_stamp: "Formatted at 2026-08-07 10:55:17 on dist-test-slave-x5fp"
server_key: "24de6befd19b38c72e6b00a1f660e40a"
server_key_iv: "68ce13e0e513d3d150ead8b376346970"
server_key_version: "encryptionkey@0"
I20260807 10:55:17.783236 30916 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 10:55:17.787376 30935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:17.789283 30916 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 10:55:17.789516 30916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-0/wal
uuid: "5fe5eee0679745b4863f20034ba77feb"
format_stamp: "Formatted at 2026-08-07 10:55:17 on dist-test-slave-x5fp"
server_key: "24de6befd19b38c72e6b00a1f660e40a"
server_key_iv: "68ce13e0e513d3d150ead8b376346970"
server_key_version: "encryptionkey@0"
I20260807 10:55:17.789749 30916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:55:17.838500 30916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:55:17.843473 30916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:55:17.843832 30916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:55:17.845062 30916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:55:17.846567 30916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:55:17.846647 30916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:17.846817 30916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:55:17.846892 30916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:17.885746 30916 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.193:43781
I20260807 10:55:17.885849 31108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.193:43781 every 8 connection(s)
I20260807 10:55:17.887214 30916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
I20260807 10:55:17.895951 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 30916
I20260807 10:55:17.896207 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance
I20260807 10:55:17.896898 3367 external_mini_cluster.cc:1428] Setting key 0ef441c5fbb112ed04412a8bdc4ace20
Aug 07 10:55:17 dist-test-slave-x5fp krb5kdc[30744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100117, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:55:17.913551 31113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:17.890694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:34843 (local address 127.3.73.254:37719)
0807 10:55:17.891276 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:17.891282 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:17.892239 (+ 957us) server_negotiation.cc:407] Connection header received
0807 10:55:17.894410 (+ 2171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:17.894415 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:17.894497 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:17.894652 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:17.896535 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:17.897657 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:17.899975 (+ 2318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:17.900693 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:17.900882 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:17.905361 (+ 4479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:17.905370 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:17.905378 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:55:17.905434 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:17.909358 (+ 3924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:17.910451 (+ 1093us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:17.910460 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:17.910466 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:17.910643 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:17.911220 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:17.911227 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:17.911233 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:17.911643 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:17.911801 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:17.912749 (+ 948us) server_negotiation.cc:299] Negotiation successful
0807 10:55:17.913054 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":291,"threads_started":1}
I20260807 10:55:17.915040 31114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:17.891159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:37719 (local address 127.3.73.193:34843)
0807 10:55:17.892035 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:17.892089 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:17.894079 (+ 1990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:17.894963 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:17.894973 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:17.895572 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:55:17.896277 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:17.896327 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:17.897920 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:17.897925 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:17.899648 (+ 1723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:17.899666 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:17.901032 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:17.901038 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:17.901310 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:17.902137 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:55:17.902164 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:55:17.905044 (+ 2880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:55:17.909620 (+ 4576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:55:17.909629 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:55:17.909646 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:55:17.910265 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:55:17.910816 (+ 551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:55:17.910820 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:55:17.910823 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:55:17.911093 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:55:17.911863 (+ 770us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:55:17.911871 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:55:17.912260 (+ 389us) client_negotiation.cc:769] Sending connection context
0807 10:55:17.912574 (+ 314us) client_negotiation.cc:240] Negotiation successful
0807 10:55:17.913145 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":340,"threads_started":1}
I20260807 10:55:17.917842 31109 heartbeater.cc:347] Connected to a master server at 127.3.73.254:37719
I20260807 10:55:17.918295 31109 heartbeater.cc:464] Registering TS with master...
I20260807 10:55:17.919171 31109 heartbeater.cc:511] Master 127.3.73.254:37719 requested a full tablet report, sending...
I20260807 10:55:17.922122 30820 ts_manager.cc:194] Registered new tserver with Master: 5fe5eee0679745b4863f20034ba77feb (127.3.73.193:43781)
I20260807 10:55:17.923774 30820 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.193@KRBTEST.COM'} at 127.3.73.193:34843
WARNING: no policy specified for kudu/127.3.73.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab.
Entry for principal kudu/127.3.73.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab.
I20260807 10:55:17.991532 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.194:0
--local_ip_for_outbound_sockets=127.3.73.194
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:37719
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:33907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:55:18.388957 31118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:55:18.389392 31118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:55:18.389596 31118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:55:18.398190 31118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:55:18.398408 31118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:55:18.398625 31118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.194
I20260807 10:55:18.409976 31118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:33907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:37719
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.194
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:55:18.411484 31118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:55:18.413378 31118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:55:18.429265 31127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:18.429414 31123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:18.429312 31129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:18.430914 31118 server_base.cc:1034] running on GCE node
I20260807 10:55:18.431712 31118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:55:18.433112 31118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:55:18.434428 31118 hybrid_clock.cc:648] HybridClock initialized: now 1786100118434317 us; error 78 us; skew 500 ppm
I20260807 10:55:18.434942 31118 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:55:18 dist-test-slave-x5fp krb5kdc[30744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100118, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:55:18.438912 31118 init.cc:377] Logged in from keytab as kudu/127.3.73.194@KRBTEST.COM (short username kudu)
I20260807 10:55:18.440955 31118 webserver.cc:460] Webserver started at http://127.3.73.194:36473/ using document root <none> and password file <none>
I20260807 10:55:18.441833 31118 fs_manager.cc:359] Metadata directory not provided
I20260807 10:55:18.442180 31118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:55:18.442601 31118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:55:18.446061 31118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-1/data/instance:
uuid: "f27986436ed7473b989baed4665babe8"
format_stamp: "Formatted at 2026-08-07 10:55:18 on dist-test-slave-x5fp"
server_key: "28a83b97497fa238b792fa89f425c080"
server_key_iv: "f7e3bbbac023c8e84b3c7eb2db2680ae"
server_key_version: "encryptionkey@0"
I20260807 10:55:18.446911 31118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance:
uuid: "f27986436ed7473b989baed4665babe8"
format_stamp: "Formatted at 2026-08-07 10:55:18 on dist-test-slave-x5fp"
server_key: "28a83b97497fa238b792fa89f425c080"
server_key_iv: "f7e3bbbac023c8e84b3c7eb2db2680ae"
server_key_version: "encryptionkey@0"
I20260807 10:55:18.453222 31118 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 10:55:18.457716 31138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:18.459477 31118 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 10:55:18.459717 31118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-1/wal
uuid: "f27986436ed7473b989baed4665babe8"
format_stamp: "Formatted at 2026-08-07 10:55:18 on dist-test-slave-x5fp"
server_key: "28a83b97497fa238b792fa89f425c080"
server_key_iv: "f7e3bbbac023c8e84b3c7eb2db2680ae"
server_key_version: "encryptionkey@0"
I20260807 10:55:18.459980 31118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:55:18.478900 31118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:55:18.484899 31118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:55:18.485292 31118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:55:18.486483 31118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:55:18.488161 31118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:55:18.488291 31118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:18.488482 31118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:55:18.488571 31118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:18.531152 31118 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.194:41503
I20260807 10:55:18.531215 31302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.194:41503 every 8 connection(s)
I20260807 10:55:18.532886 31118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
I20260807 10:55:18.535225 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 31118
I20260807 10:55:18.535485 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance
I20260807 10:55:18.536031 3367 external_mini_cluster.cc:1428] Setting key 028211bd635588129db8d0a3de0feaaa
Aug 07 10:55:18 dist-test-slave-x5fp krb5kdc[30744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100118, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:55:18.563962 31307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:18.536277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:53455 (local address 127.3.73.254:37719)
0807 10:55:18.536875 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:18.536883 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:18.538385 (+ 1502us) server_negotiation.cc:407] Connection header received
0807 10:55:18.541554 (+ 3169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:18.541559 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:18.541656 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:18.541821 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:18.544955 (+ 3134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:18.545726 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:18.548907 (+ 3181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:18.549405 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:18.549533 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:18.555754 (+ 6221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:18.555760 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:18.555763 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:55:18.555815 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:18.559763 (+ 3948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:18.560861 (+ 1098us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:18.560870 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:18.560877 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:18.561082 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:18.561634 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:18.561643 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:18.561649 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:18.562088 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:18.562274 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:18.563132 (+ 858us) server_negotiation.cc:299] Negotiation successful
0807 10:55:18.563456 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":278,"threads_started":1}
I20260807 10:55:18.565114 31309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:18.537178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:37719 (local address 127.3.73.194:53455)
0807 10:55:18.538218 (+ 1040us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:18.538289 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:18.541142 (+ 2853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:18.542171 (+ 1029us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:18.542187 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:18.543307 (+ 1120us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:55:18.544498 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:18.544538 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:18.545971 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:18.545980 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:18.548505 (+ 2525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:18.548534 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:18.549757 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:18.549770 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:18.550175 (+ 405us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:18.551219 (+ 1044us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:55:18.551271 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:55:18.555373 (+ 4102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:55:18.560015 (+ 4642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:55:18.560025 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:55:18.560046 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:55:18.560644 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:55:18.561230 (+ 586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:55:18.561233 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:55:18.561236 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:55:18.561478 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:55:18.562244 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:55:18.562251 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:55:18.562677 (+ 426us) client_negotiation.cc:769] Sending connection context
0807 10:55:18.562959 (+ 282us) client_negotiation.cc:240] Negotiation successful
0807 10:55:18.563567 (+ 608us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"thread_start_us":395,"threads_started":1}
I20260807 10:55:18.567024 31305 heartbeater.cc:347] Connected to a master server at 127.3.73.254:37719
I20260807 10:55:18.567430 31305 heartbeater.cc:464] Registering TS with master...
I20260807 10:55:18.568478 31305 heartbeater.cc:511] Master 127.3.73.254:37719 requested a full tablet report, sending...
I20260807 10:55:18.570502 30820 ts_manager.cc:194] Registered new tserver with Master: f27986436ed7473b989baed4665babe8 (127.3.73.194:41503)
I20260807 10:55:18.571410 30820 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.194@KRBTEST.COM'} at 127.3.73.194:53455
WARNING: no policy specified for kudu/127.3.73.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab.
Entry for principal kudu/127.3.73.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab.
I20260807 10:55:18.629498 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.195:0
--local_ip_for_outbound_sockets=127.3.73.195
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:37719
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:33907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 10:55:18.927474 31109 heartbeater.cc:503] Master 127.3.73.254:37719 was elected leader, sending a full tablet report...
W20260807 10:55:19.005451 31317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:55:19.005797 31317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:55:19.005901 31317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:55:19.013799 31317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:55:19.013942 31317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:55:19.014063 31317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.195
I20260807 10:55:19.024552 31317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:33907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:37719
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.195
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:55:19.026060 31317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:55:19.027568 31317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:55:19.042999 31327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:19.042889 31328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:19.042946 31332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:19.043840 31317 server_base.cc:1034] running on GCE node
I20260807 10:55:19.044543 31317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:55:19.045648 31317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:55:19.046891 31317 hybrid_clock.cc:648] HybridClock initialized: now 1786100119046804 us; error 75 us; skew 500 ppm
I20260807 10:55:19.047272 31317 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:55:19 dist-test-slave-x5fp krb5kdc[30744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100119, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:55:19.051185 31317 init.cc:377] Logged in from keytab as kudu/127.3.73.195@KRBTEST.COM (short username kudu)
I20260807 10:55:19.053267 31317 webserver.cc:460] Webserver started at http://127.3.73.195:38521/ using document root <none> and password file <none>
I20260807 10:55:19.054129 31317 fs_manager.cc:359] Metadata directory not provided
I20260807 10:55:19.054309 31317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:55:19.054695 31317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:55:19.058478 31317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-2/data/instance:
uuid: "c4b986656ce64243bac221edaa0bbd67"
format_stamp: "Formatted at 2026-08-07 10:55:19 on dist-test-slave-x5fp"
server_key: "1353d1f4ec4c549badb7027fd75a1036"
server_key_iv: "d4a245918799dbea82e753c459678ff5"
server_key_version: "encryptionkey@0"
I20260807 10:55:19.059454 31317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance:
uuid: "c4b986656ce64243bac221edaa0bbd67"
format_stamp: "Formatted at 2026-08-07 10:55:19 on dist-test-slave-x5fp"
server_key: "1353d1f4ec4c549badb7027fd75a1036"
server_key_iv: "d4a245918799dbea82e753c459678ff5"
server_key_version: "encryptionkey@0"
I20260807 10:55:19.066385 31317 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260807 10:55:19.070511 31345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:19.072250 31317 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 10:55:19.072503 31317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-2/wal
uuid: "c4b986656ce64243bac221edaa0bbd67"
format_stamp: "Formatted at 2026-08-07 10:55:19 on dist-test-slave-x5fp"
server_key: "1353d1f4ec4c549badb7027fd75a1036"
server_key_iv: "d4a245918799dbea82e753c459678ff5"
server_key_version: "encryptionkey@0"
I20260807 10:55:19.072788 31317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:55:19.095146 31317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:55:19.100373 31317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:55:19.100769 31317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:55:19.102015 31317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:55:19.103606 31317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:55:19.103727 31317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:19.103907 31317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:55:19.103991 31317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:19.143312 31317 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.195:36547
I20260807 10:55:19.143376 31501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.195:36547 every 8 connection(s)
I20260807 10:55:19.145183 31317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
I20260807 10:55:19.150117 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 31317
I20260807 10:55:19.150321 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance
I20260807 10:55:19.150833 3367 external_mini_cluster.cc:1428] Setting key 3979fbdec6667eb1879d2855fd703a1c
Aug 07 10:55:19 dist-test-slave-x5fp krb5kdc[30744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100119, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:55:19.168288 31505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:19.147982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:52261 (local address 127.3.73.254:37719)
0807 10:55:19.148510 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:19.148515 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:19.149400 (+ 885us) server_negotiation.cc:407] Connection header received
0807 10:55:19.151771 (+ 2371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:19.151776 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:19.151875 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:19.152021 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:19.154351 (+ 2330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:19.155162 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:19.156822 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:19.157227 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:19.157323 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:19.161246 (+ 3923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:19.161251 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:19.161255 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:55:19.161293 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:19.164217 (+ 2924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:19.165260 (+ 1043us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:19.165266 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:19.165269 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:19.165439 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:19.166061 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:19.166065 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:19.166068 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:19.166533 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:19.166716 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:19.167783 (+ 1067us) server_negotiation.cc:299] Negotiation successful
0807 10:55:19.167993 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":279,"threads_started":1}
I20260807 10:55:19.169396 31507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:19.148387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:37719 (local address 127.3.73.195:52261)
0807 10:55:19.149213 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:19.149266 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:19.151418 (+ 2152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:19.152477 (+ 1059us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:19.152510 (+ 33us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:19.153381 (+ 871us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:55:19.154145 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:19.154168 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:19.155318 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:19.155322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:19.156630 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:19.156644 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:19.157454 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:19.157459 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:19.157673 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:19.158453 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:55:19.158486 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:55:19.160986 (+ 2500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:55:19.164399 (+ 3413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:55:19.164423 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:55:19.164438 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:55:19.165126 (+ 688us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:55:19.165566 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:55:19.165571 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:55:19.165574 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:55:19.165878 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:55:19.166707 (+ 829us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:55:19.166714 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:55:19.167349 (+ 635us) client_negotiation.cc:769] Sending connection context
0807 10:55:19.167651 (+ 302us) client_negotiation.cc:240] Negotiation successful
0807 10:55:19.168110 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":371,"threads_started":1}
I20260807 10:55:19.170945 31503 heartbeater.cc:347] Connected to a master server at 127.3.73.254:37719
I20260807 10:55:19.171299 31503 heartbeater.cc:464] Registering TS with master...
I20260807 10:55:19.172147 31503 heartbeater.cc:511] Master 127.3.73.254:37719 requested a full tablet report, sending...
I20260807 10:55:19.173879 30820 ts_manager.cc:194] Registered new tserver with Master: c4b986656ce64243bac221edaa0bbd67 (127.3.73.195:36547)
I20260807 10:55:19.174818 30820 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.195@KRBTEST.COM'} at 127.3.73.195:52261
I20260807 10:55:19.180089 3367 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 10:55:19.199826 31505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:19.184867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52116 (local address 127.3.73.254:37719)
0807 10:55:19.185060 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:19.185065 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:19.185567 (+ 502us) server_negotiation.cc:407] Connection header received
0807 10:55:19.185722 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:19.185726 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:19.185813 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:19.185907 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:19.187542 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:19.188293 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:19.189736 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:19.190142 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:19.190248 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:19.193452 (+ 3204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:19.193458 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:19.193461 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:55:19.193503 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:19.196477 (+ 2974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:19.197609 (+ 1132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:19.197614 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:19.197616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:19.197778 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:19.198272 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:19.198275 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:19.198278 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:19.198635 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:19.198867 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:19.199394 (+ 527us) server_negotiation.cc:299] Negotiation successful
0807 10:55:19.199570 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 10:55:19.204670 30820 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52116:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 10:55:19.213227 30820 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 10:55:19.252359 31521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:19.228523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:36547 (local address 127.0.0.1:33422)
0807 10:55:19.239707 (+ 11184us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:19.239799 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:19.240140 (+ 341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:19.242960 (+ 2820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:19.242973 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:19.243176 (+ 203us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:55:19.244555 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:19.244566 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:19.246729 (+ 2163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:19.246735 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:19.249531 (+ 2796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:19.249548 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:19.251187 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:19.251197 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:19.251418 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:19.251479 (+ 61us) client_negotiation.cc:769] Sending connection context
0807 10:55:19.251649 (+ 170us) client_negotiation.cc:240] Negotiation successful
0807 10:55:19.251881 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10757,"mutex_wait_us":4,"thread_start_us":15597,"threads_started":1}
I20260807 10:55:19.253499 31522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:19.228865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33422 (local address 127.3.73.195:36547)
0807 10:55:19.241864 (+ 12999us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:19.241874 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:19.241918 (+ 44us) server_negotiation.cc:407] Connection header received
0807 10:55:19.242055 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:19.242066 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:19.242477 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:19.243618 (+ 1141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:55:19.244823 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:19.246559 (+ 1736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:19.249731 (+ 3172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:19.250966 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:19.251143 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:19.251432 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:19.251710 (+ 278us) server_negotiation.cc:299] Negotiation successful
0807 10:55:19.252983 (+ 1273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12809,"thread_start_us":12608,"threads_started":1}
I20260807 10:55:19.257263 31414 tablet_service.cc:1467] Processing CreateTablet for tablet e673a097c0004cd59bd0cfb45f74405d (DEFAULT_TABLE table=test-table [id=9fd7a370d3aa4f4ea454da4c3ec261b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:55:19.259301 31414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e673a097c0004cd59bd0cfb45f74405d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:55:19.259796 31520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:19.241950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:43781 (local address 127.0.0.1:49758)
0807 10:55:19.244457 (+ 2507us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:19.244488 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:19.244780 (+ 292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:19.245821 (+ 1041us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:19.245825 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:19.245850 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:55:19.246260 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:19.246271 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:19.247877 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:19.247882 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:19.250103 (+ 2221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:19.250120 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:19.258752 (+ 8632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:19.258762 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:19.258972 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:19.259010 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 10:55:19.259128 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 10:55:19.259302 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2353,"mutex_wait_us":28,"thread_start_us":1580,"threads_started":1}
I20260807 10:55:19.259977 31524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:19.240918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49758 (local address 127.3.73.193:43781)
0807 10:55:19.243507 (+ 2589us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:19.243518 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:19.244621 (+ 1103us) server_negotiation.cc:407] Connection header received
0807 10:55:19.245005 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:19.245015 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:19.245281 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:19.245479 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:55:19.246525 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:19.247649 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:19.250311 (+ 2662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:19.252037 (+ 1726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:19.252209 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:19.258570 (+ 6361us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:19.259241 (+ 671us) server_negotiation.cc:299] Negotiation successful
0807 10:55:19.259503 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2239,"thread_start_us":2025,"threads_started":1}
I20260807 10:55:19.260908 31523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:19.229199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58046 (local address 127.3.73.194:41503)
0807 10:55:19.241365 (+ 12166us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:19.241379 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:19.244710 (+ 3331us) server_negotiation.cc:407] Connection header received
0807 10:55:19.244888 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:19.244898 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:19.245199 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:19.245434 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:55:19.246425 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:19.248276 (+ 1851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:19.251624 (+ 3348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:19.258862 (+ 7238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:19.259039 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:19.259305 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:19.260311 (+ 1006us) server_negotiation.cc:299] Negotiation successful
0807 10:55:19.260524 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11966,"thread_start_us":11780,"threads_started":1}
I20260807 10:55:19.261144 31526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:19.228840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:41503 (local address 127.0.0.1:58046)
0807 10:55:19.244310 (+ 15470us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:19.244336 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:19.244538 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:19.245685 (+ 1147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:19.245690 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:19.245716 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:55:19.246198 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:19.246214 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:19.249006 (+ 2792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:19.249013 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:19.251381 (+ 2368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:19.251398 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:19.259508 (+ 8110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:19.259518 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:19.259743 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:19.259793 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 10:55:19.259904 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 10:55:19.260091 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15302,"thread_start_us":10145,"threads_started":1}
I20260807 10:55:19.264750 31021 tablet_service.cc:1467] Processing CreateTablet for tablet e673a097c0004cd59bd0cfb45f74405d (DEFAULT_TABLE table=test-table [id=9fd7a370d3aa4f4ea454da4c3ec261b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:55:19.265090 31204 tablet_service.cc:1467] Processing CreateTablet for tablet e673a097c0004cd59bd0cfb45f74405d (DEFAULT_TABLE table=test-table [id=9fd7a370d3aa4f4ea454da4c3ec261b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:55:19.266536 31021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e673a097c0004cd59bd0cfb45f74405d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:55:19.266535 31204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e673a097c0004cd59bd0cfb45f74405d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:55:19.273699 31527 tablet_bootstrap.cc:492] T e673a097c0004cd59bd0cfb45f74405d P c4b986656ce64243bac221edaa0bbd67: Bootstrap starting.
I20260807 10:55:19.277592 31527 tablet_bootstrap.cc:654] T e673a097c0004cd59bd0cfb45f74405d P c4b986656ce64243bac221edaa0bbd67: Neither blocks nor log segments found. Creating new log.
I20260807 10:55:19.278867 31527 log.cc:824] T e673a097c0004cd59bd0cfb45f74405d P c4b986656ce64243bac221edaa0bbd67: Log is configured to *not* fsync() on all Append() calls
I20260807 10:55:19.283762 31527 tablet_bootstrap.cc:492] T e673a097c0004cd59bd0cfb45f74405d P c4b986656ce64243bac221edaa0bbd67: No bootstrap required, opened a new log
I20260807 10:55:19.284118 31527 ts_tablet_manager.cc:1397] T e673a097c0004cd59bd0cfb45f74405d P c4b986656ce64243bac221edaa0bbd67: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260807 10:55:19.284821 31530 tablet_bootstrap.cc:492] T e673a097c0004cd59bd0cfb45f74405d P f27986436ed7473b989baed4665babe8: Bootstrap starting.
I20260807 10:55:19.285240 31529 tablet_bootstrap.cc:492] T e673a097c0004cd59bd0cfb45f74405d P 5fe5eee0679745b4863f20034ba77feb: Bootstrap starting.
I20260807 10:55:19.288906 31530 tablet_bootstrap.cc:654] T e673a097c0004cd59bd0cfb45f74405d P f27986436ed7473b989baed4665babe8: Neither blocks nor log segments found. Creating new log.
I20260807 10:55:19.289076 31529 tablet_bootstrap.cc:654] T e673a097c0004cd59bd0cfb45f74405d P 5fe5eee0679745b4863f20034ba77feb: Neither blocks nor log segments found. Creating new log.
I20260807 10:55:19.290077 31530 log.cc:824] T e673a097c0004cd59bd0cfb45f74405d P f27986436ed7473b989baed4665babe8: Log is configured to *not* fsync() on all Append() calls
I20260807 10:55:19.290345 31529 log.cc:824] T e673a097c0004cd59bd0cfb45f74405d P 5fe5eee0679745b4863f20034ba77feb: Log is configured to *not* fsync() on all Append() calls
I20260807 10:55:19.293551 31529 tablet_bootstrap.cc:492] T e673a097c0004cd59bd0cfb45f74405d P 5fe5eee0679745b4863f20034ba77feb: No bootstrap required, opened a new log
I20260807 10:55:19.293629 31530 tablet_bootstrap.cc:492] T e673a097c0004cd59bd0cfb45f74405d P f27986436ed7473b989baed4665babe8: No bootstrap required, opened a new log
I20260807 10:55:19.293918 31529 ts_tablet_manager.cc:1397] T e673a097c0004cd59bd0cfb45f74405d P 5fe5eee0679745b4863f20034ba77feb: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260807 10:55:19.293928 31530 ts_tablet_manager.cc:1397] T e673a097c0004cd59bd0cfb45f74405d P f27986436ed7473b989baed4665babe8: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.001s
I20260807 10:55:19.295236 31527 raft_consensus.cc:348] T e673a097c0004cd59bd0cfb45f74405d P c4b986656ce64243bac221edaa0bbd67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b986656ce64243bac221edaa0bbd67" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36547 } } peers { permanent_uuid: "f27986436ed7473b989baed4665babe8" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41503 } } peers { permanent_uuid: "5fe5eee0679745b4863f20034ba77feb" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43781 } }
I20260807 10:55:19.295763 31527 raft_consensus.cc:374] T e673a097c0004cd59bd0cfb45f74405d P c4b986656ce64243bac221edaa0bbd67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:55:19.295868 31527 raft_consensus.cc:729] T e673a097c0004cd59bd0cfb45f74405d P c4b986656ce64243bac221edaa0bbd67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4b986656ce64243bac221edaa0bbd67, State: Initialized, Role: FOLLOWER
I20260807 10:55:19.296551 31527 consensus_queue.cc:260] T e673a097c0004cd59bd0cfb45f74405d P c4b986656ce64243bac221edaa0bbd67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b986656ce64243bac221edaa0bbd67" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36547 } } peers { permanent_uuid: "f27986436ed7473b989baed4665babe8" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41503 } } peers { permanent_uuid: "5fe5eee0679745b4863f20034ba77feb" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43781 } }
I20260807 10:55:19.301719 31503 heartbeater.cc:503] Master 127.3.73.254:37719 was elected leader, sending a full tablet report...
I20260807 10:55:19.302747 31527 ts_tablet_manager.cc:1428] T e673a097c0004cd59bd0cfb45f74405d P c4b986656ce64243bac221edaa0bbd67: Time spent starting tablet: real 0.018s user 0.008s sys 0.012s
I20260807 10:55:19.304605 31529 raft_consensus.cc:348] T e673a097c0004cd59bd0cfb45f74405d P 5fe5eee0679745b4863f20034ba77feb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b986656ce64243bac221edaa0bbd67" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36547 } } peers { permanent_uuid: "f27986436ed7473b989baed4665babe8" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41503 } } peers { permanent_uuid: "5fe5eee0679745b4863f20034ba77feb" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43781 } }
I20260807 10:55:19.305392 31529 raft_consensus.cc:374] T e673a097c0004cd59bd0cfb45f74405d P 5fe5eee0679745b4863f20034ba77feb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:55:19.305523 31529 raft_consensus.cc:729] T e673a097c0004cd59bd0cfb45f74405d P 5fe5eee0679745b4863f20034ba77feb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fe5eee0679745b4863f20034ba77feb, State: Initialized, Role: FOLLOWER
I20260807 10:55:19.305773 31530 raft_consensus.cc:348] T e673a097c0004cd59bd0cfb45f74405d P f27986436ed7473b989baed4665babe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b986656ce64243bac221edaa0bbd67" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36547 } } peers { permanent_uuid: "f27986436ed7473b989baed4665babe8" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41503 } } peers { permanent_uuid: "5fe5eee0679745b4863f20034ba77feb" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43781 } }
I20260807 10:55:19.306252 31530 raft_consensus.cc:374] T e673a097c0004cd59bd0cfb45f74405d P f27986436ed7473b989baed4665babe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:55:19.306393 31530 raft_consensus.cc:729] T e673a097c0004cd59bd0cfb45f74405d P f27986436ed7473b989baed4665babe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f27986436ed7473b989baed4665babe8, State: Initialized, Role: FOLLOWER
I20260807 10:55:19.306260 31529 consensus_queue.cc:260] T e673a097c0004cd59bd0cfb45f74405d P 5fe5eee0679745b4863f20034ba77feb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b986656ce64243bac221edaa0bbd67" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36547 } } peers { permanent_uuid: "f27986436ed7473b989baed4665babe8" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41503 } } peers { permanent_uuid: "5fe5eee0679745b4863f20034ba77feb" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43781 } }
I20260807 10:55:19.307101 31530 consensus_queue.cc:260] T e673a097c0004cd59bd0cfb45f74405d P f27986436ed7473b989baed4665babe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b986656ce64243bac221edaa0bbd67" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36547 } } peers { permanent_uuid: "f27986436ed7473b989baed4665babe8" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41503 } } peers { permanent_uuid: "5fe5eee0679745b4863f20034ba77feb" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43781 } }
I20260807 10:55:19.308372 31529 ts_tablet_manager.cc:1428] T e673a097c0004cd59bd0cfb45f74405d P 5fe5eee0679745b4863f20034ba77feb: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260807 10:55:19.309427 31305 heartbeater.cc:503] Master 127.3.73.254:37719 was elected leader, sending a full tablet report...
I20260807 10:55:19.310024 31530 ts_tablet_manager.cc:1428] T e673a097c0004cd59bd0cfb45f74405d P f27986436ed7473b989baed4665babe8: Time spent starting tablet: real 0.016s user 0.004s sys 0.012s
I20260807 10:55:19.321674 31535 raft_consensus.cc:482] T e673a097c0004cd59bd0cfb45f74405d P c4b986656ce64243bac221edaa0bbd67 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 10:55:19.322002 31535 raft_consensus.cc:504] T e673a097c0004cd59bd0cfb45f74405d P c4b986656ce64243bac221edaa0bbd67 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b986656ce64243bac221edaa0bbd67" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36547 } } peers { permanent_uuid: "f27986436ed7473b989baed4665babe8" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41503 } } peers { permanent_uuid: "5fe5eee0679745b4863f20034ba77feb" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43781 } }
I20260807 10:55:19.323863 31535 leader_election.cc:288] T e673a097c0004cd59bd0cfb45f74405d P c4b986656ce64243bac221edaa0bbd67 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f27986436ed7473b989baed4665babe8 (127.3.73.194:41503), 5fe5eee0679745b4863f20034ba77feb (127.3.73.193:43781)
I20260807 10:55:19.331842 31523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:19.324380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:58859 (local address 127.3.73.194:41503)
0807 10:55:19.324658 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:19.324667 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:19.324693 (+ 26us) server_negotiation.cc:407] Connection header received
0807 10:55:19.325074 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:19.325079 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:19.325149 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:19.325282 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:55:19.326034 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:19.327105 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:19.329545 (+ 2440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:19.330707 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:19.330921 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:19.331010 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:19.331354 (+ 344us) server_negotiation.cc:299] Negotiation successful
0807 10:55:19.331505 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260807 10:55:19.331892 31507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:19.324371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:41503 (local address 127.3.73.195:58859)
0807 10:55:19.324608 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:19.324628 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:19.324860 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:19.325541 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:19.325549 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:19.325588 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:55:19.325844 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:19.325862 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:19.327308 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:19.327314 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:19.329320 (+ 2006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:19.329361 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:19.330901 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:19.330910 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:19.331153 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:19.331183 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 10:55:19.331321 (+ 138us) client_negotiation.cc:240] Negotiation successful
0807 10:55:19.331491 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260807 10:55:19.331988 31524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:19.325179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:60435 (local address 127.3.73.193:43781)
0807 10:55:19.325465 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:19.325471 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:19.325910 (+ 439us) server_negotiation.cc:407] Connection header received
0807 10:55:19.326036 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:19.326040 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:19.326129 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:19.326318 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:55:19.326919 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:19.327888 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:19.330094 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:19.330922 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:19.331143 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:19.331231 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:19.331547 (+ 316us) server_negotiation.cc:299] Negotiation successful
0807 10:55:19.331691 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260807 10:55:19.332674 31540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:19.324885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:43781 (local address 127.3.73.195:60435)
0807 10:55:19.325751 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:19.325780 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:19.325922 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:19.326302 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:19.326305 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:19.326329 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:55:19.326726 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:19.326739 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:19.328093 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:19.328123 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:19.329881 (+ 1758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:19.329900 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:19.331165 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:19.331172 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:19.331365 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:19.331391 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 10:55:19.331510 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 10:55:19.331654 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"mutex_wait_us":55,"thread_start_us":458,"threads_started":1}
I20260807 10:55:19.333472 31235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e673a097c0004cd59bd0cfb45f74405d" candidate_uuid: "c4b986656ce64243bac221edaa0bbd67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f27986436ed7473b989baed4665babe8" is_pre_election: true
I20260807 10:55:19.333969 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e673a097c0004cd59bd0cfb45f74405d" candidate_uuid: "c4b986656ce64243bac221edaa0bbd67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fe5eee0679745b4863f20034ba77feb" is_pre_election: true
I20260807 10:55:19.334095 31235 raft_consensus.cc:2454] T e673a097c0004cd59bd0cfb45f74405d P f27986436ed7473b989baed4665babe8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4b986656ce64243bac221edaa0bbd67 in term 0.
I20260807 10:55:19.334507 31054 raft_consensus.cc:2454] T e673a097c0004cd59bd0cfb45f74405d P 5fe5eee0679745b4863f20034ba77feb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4b986656ce64243bac221edaa0bbd67 in term 0.
I20260807 10:55:19.334903 31348 leader_election.cc:302] T e673a097c0004cd59bd0cfb45f74405d P c4b986656ce64243bac221edaa0bbd67 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b986656ce64243bac221edaa0bbd67, f27986436ed7473b989baed4665babe8; no voters:
I20260807 10:55:19.335407 31535 raft_consensus.cc:2781] T e673a097c0004cd59bd0cfb45f74405d P c4b986656ce64243bac221edaa0bbd67 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 10:55:19.335580 31535 raft_consensus.cc:482] T e673a097c0004cd59bd0cfb45f74405d P c4b986656ce64243bac221edaa0bbd67 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 10:55:19.335721 31535 raft_consensus.cc:3037] T e673a097c0004cd59bd0cfb45f74405d P c4b986656ce64243bac221edaa0bbd67 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:55:19.338423 31535 raft_consensus.cc:504] T e673a097c0004cd59bd0cfb45f74405d P c4b986656ce64243bac221edaa0bbd67 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b986656ce64243bac221edaa0bbd67" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36547 } } peers { permanent_uuid: "f27986436ed7473b989baed4665babe8" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41503 } } peers { permanent_uuid: "5fe5eee0679745b4863f20034ba77feb" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43781 } }
I20260807 10:55:19.339185 31535 leader_election.cc:288] T e673a097c0004cd59bd0cfb45f74405d P c4b986656ce64243bac221edaa0bbd67 [CANDIDATE]: Term 1 election: Requested vote from peers f27986436ed7473b989baed4665babe8 (127.3.73.194:41503), 5fe5eee0679745b4863f20034ba77feb (127.3.73.193:43781)
I20260807 10:55:19.340009 31235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e673a097c0004cd59bd0cfb45f74405d" candidate_uuid: "c4b986656ce64243bac221edaa0bbd67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f27986436ed7473b989baed4665babe8"
I20260807 10:55:19.340114 31054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e673a097c0004cd59bd0cfb45f74405d" candidate_uuid: "c4b986656ce64243bac221edaa0bbd67" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fe5eee0679745b4863f20034ba77feb"
I20260807 10:55:19.340265 31235 raft_consensus.cc:3037] T e673a097c0004cd59bd0cfb45f74405d P f27986436ed7473b989baed4665babe8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:55:19.340431 31054 raft_consensus.cc:3037] T e673a097c0004cd59bd0cfb45f74405d P 5fe5eee0679745b4863f20034ba77feb [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:55:19.342823 31235 raft_consensus.cc:2454] T e673a097c0004cd59bd0cfb45f74405d P f27986436ed7473b989baed4665babe8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4b986656ce64243bac221edaa0bbd67 in term 1.
I20260807 10:55:19.342847 31054 raft_consensus.cc:2454] T e673a097c0004cd59bd0cfb45f74405d P 5fe5eee0679745b4863f20034ba77feb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4b986656ce64243bac221edaa0bbd67 in term 1.
I20260807 10:55:19.343415 31348 leader_election.cc:302] T e673a097c0004cd59bd0cfb45f74405d P c4b986656ce64243bac221edaa0bbd67 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c4b986656ce64243bac221edaa0bbd67, f27986436ed7473b989baed4665babe8; no voters:
I20260807 10:55:19.343815 31535 raft_consensus.cc:2781] T e673a097c0004cd59bd0cfb45f74405d P c4b986656ce64243bac221edaa0bbd67 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:55:19.344368 31535 raft_consensus.cc:686] T e673a097c0004cd59bd0cfb45f74405d P c4b986656ce64243bac221edaa0bbd67 [term 1 LEADER]: Becoming Leader. State: Replica: c4b986656ce64243bac221edaa0bbd67, State: Running, Role: LEADER
I20260807 10:55:19.345119 31535 consensus_queue.cc:237] T e673a097c0004cd59bd0cfb45f74405d P c4b986656ce64243bac221edaa0bbd67 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b986656ce64243bac221edaa0bbd67" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36547 } } peers { permanent_uuid: "f27986436ed7473b989baed4665babe8" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41503 } } peers { permanent_uuid: "5fe5eee0679745b4863f20034ba77feb" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43781 } }
I20260807 10:55:19.350497 30820 catalog_manager.cc:5495] T e673a097c0004cd59bd0cfb45f74405d P c4b986656ce64243bac221edaa0bbd67 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4b986656ce64243bac221edaa0bbd67 (127.3.73.195). New cstate: current_term: 1 leader_uuid: "c4b986656ce64243bac221edaa0bbd67" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c4b986656ce64243bac221edaa0bbd67" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 36547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f27986436ed7473b989baed4665babe8" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fe5eee0679745b4863f20034ba77feb" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43781 } health_report { overall_health: UNKNOWN } } }
I20260807 10:55:19.388324 31522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:19.382160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33430 (local address 127.3.73.195:36547)
0807 10:55:19.382427 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:19.382431 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:19.382450 (+ 19us) server_negotiation.cc:407] Connection header received
0807 10:55:19.382542 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:19.382546 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:19.382627 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:19.382769 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:55:19.383266 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:19.384001 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:19.385645 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:19.386075 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:19.386245 (+ 170us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:19.386908 (+ 663us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:55:19.387612 (+ 704us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:55:19.387817 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:19.387995 (+ 178us) server_negotiation.cc:299] Negotiation successful
0807 10:55:19.388047 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260807 10:55:19.390331 31111 tablet.cc:2369] T e673a097c0004cd59bd0cfb45f74405d P 5fe5eee0679745b4863f20034ba77feb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 10:55:19.396968 31504 tablet.cc:2369] T e673a097c0004cd59bd0cfb45f74405d P c4b986656ce64243bac221edaa0bbd67: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:55:19.397419 31054 raft_consensus.cc:1261] T e673a097c0004cd59bd0cfb45f74405d P 5fe5eee0679745b4863f20034ba77feb [term 1 FOLLOWER]: Refusing update from remote peer c4b986656ce64243bac221edaa0bbd67: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:55:19.397570 31235 raft_consensus.cc:1261] T e673a097c0004cd59bd0cfb45f74405d P f27986436ed7473b989baed4665babe8 [term 1 FOLLOWER]: Refusing update from remote peer c4b986656ce64243bac221edaa0bbd67: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:55:19.398547 31541 consensus_queue.cc:1035] T e673a097c0004cd59bd0cfb45f74405d P c4b986656ce64243bac221edaa0bbd67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fe5eee0679745b4863f20034ba77feb" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 43781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:55:19.398986 31535 consensus_queue.cc:1035] T e673a097c0004cd59bd0cfb45f74405d P c4b986656ce64243bac221edaa0bbd67 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f27986436ed7473b989baed4665babe8" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 41503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:55:19.412314 31547 mvcc.cc:205] Tried to move back new op lower bound from 7315866089040261120 to 7315866088842170368. Current Snapshot: MvccSnapshot[applied={T|T < 7315866089040261120}]
I20260807 10:55:19.430487 30819 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:55:19.430860 30819 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52116:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:55:19.435149 30819 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fc852db596474e50b9ba0aa802770eb8: Sending DeleteTablet for 3 replicas of tablet e673a097c0004cd59bd0cfb45f74405d
I20260807 10:55:19.436590 31414 tablet_service.cc:1514] Processing DeleteTablet for tablet e673a097c0004cd59bd0cfb45f74405d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:55:19 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:33422
I20260807 10:55:19.436753 31204 tablet_service.cc:1514] Processing DeleteTablet for tablet e673a097c0004cd59bd0cfb45f74405d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:55:19 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:58046
I20260807 10:55:19.437714 31559 tablet_replica.cc:330] T e673a097c0004cd59bd0cfb45f74405d P c4b986656ce64243bac221edaa0bbd67: stopping tablet replica
I20260807 10:55:19.438289 31559 raft_consensus.cc:2229] T e673a097c0004cd59bd0cfb45f74405d P c4b986656ce64243bac221edaa0bbd67 [term 1 LEADER]: Raft consensus shutting down.
I20260807 10:55:19.438534 31021 tablet_service.cc:1514] Processing DeleteTablet for tablet e673a097c0004cd59bd0cfb45f74405d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:55:19 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:49758
I20260807 10:55:19.439258 31559 raft_consensus.cc:2258] T e673a097c0004cd59bd0cfb45f74405d P c4b986656ce64243bac221edaa0bbd67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:55:19.441301 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 30916
I20260807 10:55:19.441995 31559 ts_tablet_manager.cc:1905] T e673a097c0004cd59bd0cfb45f74405d P c4b986656ce64243bac221edaa0bbd67: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:55:19.445340 31560 tablet_replica.cc:330] T e673a097c0004cd59bd0cfb45f74405d P f27986436ed7473b989baed4665babe8: stopping tablet replica
I20260807 10:55:19.445948 31560 raft_consensus.cc:2229] T e673a097c0004cd59bd0cfb45f74405d P f27986436ed7473b989baed4665babe8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:55:19.446538 31560 raft_consensus.cc:2258] T e673a097c0004cd59bd0cfb45f74405d P f27986436ed7473b989baed4665babe8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:55:19.450089 31562 tablet_replica.cc:330] T e673a097c0004cd59bd0cfb45f74405d P 5fe5eee0679745b4863f20034ba77feb: stopping tablet replica
I20260807 10:55:19.450717 31562 raft_consensus.cc:2229] T e673a097c0004cd59bd0cfb45f74405d P 5fe5eee0679745b4863f20034ba77feb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:55:19.451434 31562 raft_consensus.cc:2258] T e673a097c0004cd59bd0cfb45f74405d P 5fe5eee0679745b4863f20034ba77feb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:55:19.454286 31562 ts_tablet_manager.cc:1905] T e673a097c0004cd59bd0cfb45f74405d P 5fe5eee0679745b4863f20034ba77feb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:55:19.454564 31560 ts_tablet_manager.cc:1905] T e673a097c0004cd59bd0cfb45f74405d P f27986436ed7473b989baed4665babe8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:55:19.452942 31559 ts_tablet_manager.cc:1918] T e673a097c0004cd59bd0cfb45f74405d P c4b986656ce64243bac221edaa0bbd67: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:55:19.457123 31559 log.cc:1196] T e673a097c0004cd59bd0cfb45f74405d P c4b986656ce64243bac221edaa0bbd67: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-2/wal/wals/e673a097c0004cd59bd0cfb45f74405d
I20260807 10:55:19.457836 31559 ts_tablet_manager.cc:1939] T e673a097c0004cd59bd0cfb45f74405d P c4b986656ce64243bac221edaa0bbd67: Deleting consensus metadata
I20260807 10:55:19.460160 30796 catalog_manager.cc:4841] TS c4b986656ce64243bac221edaa0bbd67 (127.3.73.195:36547): tablet e673a097c0004cd59bd0cfb45f74405d (table test-table [id=9fd7a370d3aa4f4ea454da4c3ec261b0]) successfully deleted
I20260807 10:55:19.462813 31560 ts_tablet_manager.cc:1918] T e673a097c0004cd59bd0cfb45f74405d P f27986436ed7473b989baed4665babe8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:55:19.463042 31560 log.cc:1196] T e673a097c0004cd59bd0cfb45f74405d P f27986436ed7473b989baed4665babe8: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-1/wal/wals/e673a097c0004cd59bd0cfb45f74405d
Aug 07 10:55:19 dist-test-slave-x5fp krb5kdc[30744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.193@KRBTEST.COM
I20260807 10:55:19.463668 31560 ts_tablet_manager.cc:1939] T e673a097c0004cd59bd0cfb45f74405d P f27986436ed7473b989baed4665babe8: Deleting consensus metadata
I20260807 10:55:19.464208 31562 ts_tablet_manager.cc:1918] T e673a097c0004cd59bd0cfb45f74405d P 5fe5eee0679745b4863f20034ba77feb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:55:19.464493 31562 log.cc:1196] T e673a097c0004cd59bd0cfb45f74405d P 5fe5eee0679745b4863f20034ba77feb: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786099767749326-3367-0/minicluster-data/ts-0/wal/wals/e673a097c0004cd59bd0cfb45f74405d
I20260807 10:55:19.465211 31562 ts_tablet_manager.cc:1939] T e673a097c0004cd59bd0cfb45f74405d P 5fe5eee0679745b4863f20034ba77feb: Deleting consensus metadata
I20260807 10:55:19.466037 30795 catalog_manager.cc:4841] TS f27986436ed7473b989baed4665babe8 (127.3.73.194:41503): tablet e673a097c0004cd59bd0cfb45f74405d (table test-table [id=9fd7a370d3aa4f4ea454da4c3ec261b0]) successfully deleted
I20260807 10:55:19.467751 30795 catalog_manager.cc:4841] TS 5fe5eee0679745b4863f20034ba77feb (127.3.73.193:43781): tablet e673a097c0004cd59bd0cfb45f74405d (table test-table [id=9fd7a370d3aa4f4ea454da4c3ec261b0]) successfully deleted
I20260807 10:55:19.472352 31524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:19.443402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49772 (local address 127.3.73.193:43781)
0807 10:55:19.443656 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:19.443665 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:19.449347 (+ 5682us) server_negotiation.cc:407] Connection header received
0807 10:55:19.449570 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:19.449581 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:19.449715 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:19.449843 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:19.452022 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:19.453253 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:19.455264 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:19.455930 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:19.456044 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:19.465230 (+ 9186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:19.465249 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:19.465270 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:55:19.465346 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:19.469105 (+ 3759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:19.469998 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:19.470006 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:19.470009 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:19.470161 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:19.470617 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:19.470623 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:19.470628 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:19.471034 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:19.471373 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:19.471717 (+ 344us) server_negotiation.cc:299] Negotiation successful
0807 10:55:19.471966 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 10:55:19.473115 31100 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:55:19.607671 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 30916
I20260807 10:55:19.643121 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 31118
Aug 07 10:55:19 dist-test-slave-x5fp krb5kdc[30744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.194@KRBTEST.COM
I20260807 10:55:19.665865 31523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:19.644386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58050 (local address 127.3.73.194:41503)
0807 10:55:19.644909 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:19.644920 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:19.644959 (+ 39us) server_negotiation.cc:407] Connection header received
0807 10:55:19.645043 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:19.645050 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:19.645186 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:19.645331 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:19.646893 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:19.648144 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:19.649812 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:19.650465 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:19.650622 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:19.657321 (+ 6699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:19.657350 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:19.657374 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:55:19.657440 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:19.661923 (+ 4483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:19.663161 (+ 1238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:19.663170 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:19.663173 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:19.663357 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:19.663959 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:19.663964 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:19.663967 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:19.664470 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:19.664946 (+ 476us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:19.665173 (+ 227us) server_negotiation.cc:299] Negotiation successful
0807 10:55:19.665398 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228}
I20260807 10:55:19.667130 31295 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:55:19.773201 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 31118
I20260807 10:55:19.807080 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 31317
Aug 07 10:55:19 dist-test-slave-x5fp krb5kdc[30744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.195@KRBTEST.COM
I20260807 10:55:19.830608 31522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:19.808332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33444 (local address 127.3.73.195:36547)
0807 10:55:19.808570 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:19.808575 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:19.808599 (+ 24us) server_negotiation.cc:407] Connection header received
0807 10:55:19.808678 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:19.808683 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:19.808807 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:19.808943 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:19.810293 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:19.811563 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:19.813373 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:19.814103 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:19.814283 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:19.820192 (+ 5909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:19.820210 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:19.820232 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:55:19.820302 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:19.825467 (+ 5165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:19.827048 (+ 1581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:19.827058 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:19.827060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:19.827315 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:19.828019 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:19.828027 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:19.828030 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:19.829072 (+ 1042us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:19.829396 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:19.830006 (+ 610us) server_negotiation.cc:299] Negotiation successful
0807 10:55:19.830223 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260807 10:55:19.831688 31495 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:55:19.962935 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 31317
I20260807 10:55:19.994529 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 30769
I20260807 10:55:19.995803 30874 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:55:20.111625 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 30769
2026-08-07T10:55:20Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (5329 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T10:55:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:55:20Z Disabled control of system clock
I20260807 10:55:20.224536 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:44099
--webserver_interface=127.3.73.254
--webserver_port=0
--builtin_ntp_servers=127.3.73.212:43693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:44099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 10:55:20.638109 31573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:55:20.638479 31573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:55:20.638649 31573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:55:20.646644 31573 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:55:20.646796 31573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:55:20.646890 31573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:55:20.647020 31573 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:55:20.647114 31573 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:55:20.657400 31573 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:43693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:44099
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:44099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:55:20.658918 31573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:55:20.661566 31573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:55:20.676417 31580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:20.676534 31579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:20.676522 31583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:20.678013 31573 server_base.cc:1034] running on GCE node
I20260807 10:55:20.678964 31573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:55:20.680577 31573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:55:20.681823 31573 hybrid_clock.cc:648] HybridClock initialized: now 1786100120681768 us; error 54 us; skew 500 ppm
I20260807 10:55:20.682235 31573 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 10:55:20.684902 31573 webserver.cc:460] Webserver started at http://127.3.73.254:43875/ using document root <none> and password file <none>
I20260807 10:55:20.685676 31573 fs_manager.cc:359] Metadata directory not provided
I20260807 10:55:20.685822 31573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:55:20.686184 31573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:55:20.689478 31573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "17e9fb6923b24fce9666abdcfbf1c30b"
format_stamp: "Formatted at 2026-08-07 10:55:20 on dist-test-slave-x5fp"
server_key: "e688147ecf7fc0ae530ecf263582f0c6"
server_key_iv: "106cebd9c57998ba186f5f3de64deedc"
server_key_version: "encryptionkey@0"
I20260807 10:55:20.690346 31573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "17e9fb6923b24fce9666abdcfbf1c30b"
format_stamp: "Formatted at 2026-08-07 10:55:20 on dist-test-slave-x5fp"
server_key: "e688147ecf7fc0ae530ecf263582f0c6"
server_key_iv: "106cebd9c57998ba186f5f3de64deedc"
server_key_version: "encryptionkey@0"
I20260807 10:55:20.696110 31573 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260807 10:55:20.699946 31595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:20.701465 31573 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 10:55:20.701675 31573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "17e9fb6923b24fce9666abdcfbf1c30b"
format_stamp: "Formatted at 2026-08-07 10:55:20 on dist-test-slave-x5fp"
server_key: "e688147ecf7fc0ae530ecf263582f0c6"
server_key_iv: "106cebd9c57998ba186f5f3de64deedc"
server_key_version: "encryptionkey@0"
I20260807 10:55:20.701931 31573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:55:20.726923 31573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:55:20.727946 31573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:55:20.728291 31573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:55:20.749918 31573 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:44099
I20260807 10:55:20.749955 31668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:44099 every 8 connection(s)
I20260807 10:55:20.751508 31573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:55:20.755874 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 31573
I20260807 10:55:20.756089 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:55:20.756376 31670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:55:20.756636 3367 external_mini_cluster.cc:1428] Setting key cca23e54e555ea847924e50c1fa8daec
I20260807 10:55:20.770733 31670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17e9fb6923b24fce9666abdcfbf1c30b: Bootstrap starting.
I20260807 10:55:20.772563 31675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:20.758790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58846 (local address 127.3.73.254:44099)
0807 10:55:20.759943 (+ 1153us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:20.759962 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:20.760012 (+ 50us) server_negotiation.cc:407] Connection header received
0807 10:55:20.762315 (+ 2303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:20.762344 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:20.762940 (+ 596us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:20.763445 (+ 505us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:20.764163 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:20.765805 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:20.767423 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:20.768274 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:20.768532 (+ 258us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:20.769980 (+ 1448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:20.769988 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:20.770004 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:20.770626 (+ 622us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:20.770719 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:20.770742 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:20.770900 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:20.771148 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 10:55:20.771506 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":406,"threads_started":1}
I20260807 10:55:20.774935 31670 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17e9fb6923b24fce9666abdcfbf1c30b: Neither blocks nor log segments found. Creating new log.
I20260807 10:55:20.776219 31670 log.cc:824] T 00000000000000000000000000000000 P 17e9fb6923b24fce9666abdcfbf1c30b: Log is configured to *not* fsync() on all Append() calls
I20260807 10:55:20.779573 31670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17e9fb6923b24fce9666abdcfbf1c30b: No bootstrap required, opened a new log
I20260807 10:55:20.787842 31670 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17e9fb6923b24fce9666abdcfbf1c30b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17e9fb6923b24fce9666abdcfbf1c30b" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 44099 } }
I20260807 10:55:20.788291 31670 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17e9fb6923b24fce9666abdcfbf1c30b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:55:20.788407 31670 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17e9fb6923b24fce9666abdcfbf1c30b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e9fb6923b24fce9666abdcfbf1c30b, State: Initialized, Role: FOLLOWER
I20260807 10:55:20.789166 31670 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17e9fb6923b24fce9666abdcfbf1c30b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17e9fb6923b24fce9666abdcfbf1c30b" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 44099 } }
I20260807 10:55:20.789446 31670 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17e9fb6923b24fce9666abdcfbf1c30b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:55:20.789613 31670 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17e9fb6923b24fce9666abdcfbf1c30b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:55:20.789803 31670 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17e9fb6923b24fce9666abdcfbf1c30b [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:55:20.792111 31670 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17e9fb6923b24fce9666abdcfbf1c30b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17e9fb6923b24fce9666abdcfbf1c30b" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 44099 } }
I20260807 10:55:20.792842 31670 leader_election.cc:302] T 00000000000000000000000000000000 P 17e9fb6923b24fce9666abdcfbf1c30b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17e9fb6923b24fce9666abdcfbf1c30b; no voters:
I20260807 10:55:20.793503 31670 leader_election.cc:288] T 00000000000000000000000000000000 P 17e9fb6923b24fce9666abdcfbf1c30b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:55:20.793707 31677 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17e9fb6923b24fce9666abdcfbf1c30b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:55:20.795238 31677 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17e9fb6923b24fce9666abdcfbf1c30b [term 1 LEADER]: Becoming Leader. State: Replica: 17e9fb6923b24fce9666abdcfbf1c30b, State: Running, Role: LEADER
I20260807 10:55:20.795893 31677 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17e9fb6923b24fce9666abdcfbf1c30b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17e9fb6923b24fce9666abdcfbf1c30b" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 44099 } }
I20260807 10:55:20.796053 31670 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17e9fb6923b24fce9666abdcfbf1c30b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:55:20.799600 31681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17e9fb6923b24fce9666abdcfbf1c30b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17e9fb6923b24fce9666abdcfbf1c30b. Latest consensus state: current_term: 1 leader_uuid: "17e9fb6923b24fce9666abdcfbf1c30b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17e9fb6923b24fce9666abdcfbf1c30b" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 44099 } } }
I20260807 10:55:20.799685 31678 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17e9fb6923b24fce9666abdcfbf1c30b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17e9fb6923b24fce9666abdcfbf1c30b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17e9fb6923b24fce9666abdcfbf1c30b" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 44099 } } }
I20260807 10:55:20.800367 31678 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17e9fb6923b24fce9666abdcfbf1c30b [sys.catalog]: This master's current role is: LEADER
I20260807 10:55:20.800374 31681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17e9fb6923b24fce9666abdcfbf1c30b [sys.catalog]: This master's current role is: LEADER
I20260807 10:55:20.812436 31691 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:55:20.818727 31691 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:55:20.829910 31691 catalog_manager.cc:1347] Generated new cluster ID: c16745d4e7de49a9a89543351c762794
I20260807 10:55:20.830107 31691 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:55:20.847347 31691 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:55:20.848920 31691 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:55:20.856814 31691 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17e9fb6923b24fce9666abdcfbf1c30b: Generated new TSK 0
I20260807 10:55:20.857971 31691 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 10:55:20.879426 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.193:0
--local_ip_for_outbound_sockets=127.3.73.193
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:44099
--builtin_ntp_servers=127.3.73.212:43693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 10:55:21.279981 31704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:55:21.280328 31704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:55:21.280491 31704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:55:21.288424 31704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:55:21.288559 31704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:55:21.288807 31704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.193
I20260807 10:55:21.298944 31704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:43693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:44099
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.193
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:55:21.300447 31704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:55:21.302451 31704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:55:21.317461 31710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:21.317677 31713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:21.317958 31709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:21.318535 31704 server_base.cc:1034] running on GCE node
I20260807 10:55:21.319594 31704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:55:21.320703 31704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:55:21.322032 31704 hybrid_clock.cc:648] HybridClock initialized: now 1786100121321971 us; error 83 us; skew 500 ppm
I20260807 10:55:21.322428 31704 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 10:55:21.325019 31704 webserver.cc:460] Webserver started at http://127.3.73.193:33417/ using document root <none> and password file <none>
I20260807 10:55:21.325803 31704 fs_manager.cc:359] Metadata directory not provided
I20260807 10:55:21.325948 31704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:55:21.326305 31704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:55:21.329559 31704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-0/data/instance:
uuid: "4b3d2d8691e24946820c8ccb3dbdcb8e"
format_stamp: "Formatted at 2026-08-07 10:55:21 on dist-test-slave-x5fp"
server_key: "a91882ab4d44ffd88012729a86bdc911"
server_key_iv: "51b3498b272ad96b28b4670b6054701f"
server_key_version: "encryptionkey@0"
I20260807 10:55:21.330476 31704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance:
uuid: "4b3d2d8691e24946820c8ccb3dbdcb8e"
format_stamp: "Formatted at 2026-08-07 10:55:21 on dist-test-slave-x5fp"
server_key: "a91882ab4d44ffd88012729a86bdc911"
server_key_iv: "51b3498b272ad96b28b4670b6054701f"
server_key_version: "encryptionkey@0"
I20260807 10:55:21.336556 31704 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 10:55:21.340922 31719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:21.342592 31704 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 10:55:21.342876 31704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-0/wal
uuid: "4b3d2d8691e24946820c8ccb3dbdcb8e"
format_stamp: "Formatted at 2026-08-07 10:55:21 on dist-test-slave-x5fp"
server_key: "a91882ab4d44ffd88012729a86bdc911"
server_key_iv: "51b3498b272ad96b28b4670b6054701f"
server_key_version: "encryptionkey@0"
I20260807 10:55:21.343191 31704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:55:21.356925 31704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:55:21.357869 31704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:55:21.358214 31704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:55:21.359316 31704 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:55:21.361079 31704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:55:21.361212 31704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:21.361368 31704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:55:21.361464 31704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:21.399384 31704 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.193:33881
I20260807 10:55:21.399431 31871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.193:33881 every 8 connection(s)
I20260807 10:55:21.400993 31704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
I20260807 10:55:21.406327 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 31704
I20260807 10:55:21.406540 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance
I20260807 10:55:21.407176 3367 external_mini_cluster.cc:1428] Setting key 8332a881676ed5f2aa3858b0ac97e33b
I20260807 10:55:21.410794 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.194:0
--local_ip_for_outbound_sockets=127.3.73.194
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:44099
--builtin_ntp_servers=127.3.73.212:43693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 10:55:21.417778 31875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:21.404143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:48977 (local address 127.3.73.254:44099)
0807 10:55:21.404685 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:21.404689 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:21.406337 (+ 1648us) server_negotiation.cc:407] Connection header received
0807 10:55:21.408864 (+ 2527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:21.408869 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:21.408941 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:21.409030 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:21.410624 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:21.411545 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:21.414295 (+ 2750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:21.414765 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:21.414900 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:21.416855 (+ 1955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:21.416861 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:21.416864 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:21.416890 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:21.416936 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:21.416949 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:21.417032 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:21.417365 (+ 333us) server_negotiation.cc:299] Negotiation successful
0807 10:55:21.417480 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":282,"threads_started":1}
I20260807 10:55:21.419377 31876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:21.404906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:44099 (local address 127.3.73.193:48977)
0807 10:55:21.406094 (+ 1188us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:21.406152 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:21.408478 (+ 2326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:21.409339 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:21.409349 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:21.409459 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:55:21.410406 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:21.410438 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:21.411729 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:21.411736 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:21.414103 (+ 2367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:21.414125 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:21.415048 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:21.415059 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:21.415433 (+ 374us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:21.416336 (+ 903us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 10:55:21.416342 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:55:21.416410 (+ 68us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 10:55:21.416532 (+ 122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:55:21.417063 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:55:21.417079 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:55:21.417092 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 10:55:21.417330 (+ 238us) client_negotiation.cc:240] Negotiation successful
0807 10:55:21.417827 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":322,"threads_started":1}
I20260807 10:55:21.421569 31872 heartbeater.cc:347] Connected to a master server at 127.3.73.254:44099
I20260807 10:55:21.421957 31872 heartbeater.cc:464] Registering TS with master...
I20260807 10:55:21.422968 31872 heartbeater.cc:511] Master 127.3.73.254:44099 requested a full tablet report, sending...
I20260807 10:55:21.425630 31615 ts_manager.cc:194] Registered new tserver with Master: 4b3d2d8691e24946820c8ccb3dbdcb8e (127.3.73.193:33881)
I20260807 10:55:21.427695 31615 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.73.193:48977
W20260807 10:55:21.796439 31878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:55:21.796878 31878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:55:21.797055 31878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:55:21.805017 31878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:55:21.805163 31878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:55:21.805339 31878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.194
I20260807 10:55:21.815147 31878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:43693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:44099
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.194
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:55:21.816690 31878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:55:21.818701 31878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:55:21.833590 31885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:21.833563 31886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:21.834169 31892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:21.834955 31878 server_base.cc:1034] running on GCE node
I20260807 10:55:21.835702 31878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:55:21.836921 31878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:55:21.838182 31878 hybrid_clock.cc:648] HybridClock initialized: now 1786100121838130 us; error 48 us; skew 500 ppm
I20260807 10:55:21.838568 31878 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 10:55:21.841332 31878 webserver.cc:460] Webserver started at http://127.3.73.194:35779/ using document root <none> and password file <none>
I20260807 10:55:21.842118 31878 fs_manager.cc:359] Metadata directory not provided
I20260807 10:55:21.842278 31878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:55:21.842640 31878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:55:21.846275 31878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-1/data/instance:
uuid: "54d7719a109c43588c7364ade59bb009"
format_stamp: "Formatted at 2026-08-07 10:55:21 on dist-test-slave-x5fp"
server_key: "055c771c905b508ec1759920819b209c"
server_key_iv: "6e88f51b233edc7cdaabd80dcfee710e"
server_key_version: "encryptionkey@0"
I20260807 10:55:21.847262 31878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance:
uuid: "54d7719a109c43588c7364ade59bb009"
format_stamp: "Formatted at 2026-08-07 10:55:21 on dist-test-slave-x5fp"
server_key: "055c771c905b508ec1759920819b209c"
server_key_iv: "6e88f51b233edc7cdaabd80dcfee710e"
server_key_version: "encryptionkey@0"
I20260807 10:55:21.853155 31878 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 10:55:21.857403 31900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:21.859076 31878 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 10:55:21.859303 31878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-1/wal
uuid: "54d7719a109c43588c7364ade59bb009"
format_stamp: "Formatted at 2026-08-07 10:55:21 on dist-test-slave-x5fp"
server_key: "055c771c905b508ec1759920819b209c"
server_key_iv: "6e88f51b233edc7cdaabd80dcfee710e"
server_key_version: "encryptionkey@0"
I20260807 10:55:21.859577 31878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:55:21.888185 31878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:55:21.889292 31878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:55:21.889652 31878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:55:21.890862 31878 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:55:21.892639 31878 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:55:21.892812 31878 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260807 10:55:21.892957 31878 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:55:21.893056 31878 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:21.932125 31878 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.194:42079
I20260807 10:55:21.932246 32075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.194:42079 every 8 connection(s)
I20260807 10:55:21.933669 31878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
I20260807 10:55:21.942816 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 31878
I20260807 10:55:21.943010 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance
I20260807 10:55:21.943518 3367 external_mini_cluster.cc:1428] Setting key 2f765d36ba717aa4eb5fb30aabb10ab6
I20260807 10:55:21.946766 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.195:0
--local_ip_for_outbound_sockets=127.3.73.195
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:44099
--builtin_ntp_servers=127.3.73.212:43693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 10:55:21.947950 32078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:21.936585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:46555 (local address 127.3.73.254:44099)
0807 10:55:21.937188 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:21.937193 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:21.938429 (+ 1236us) server_negotiation.cc:407] Connection header received
0807 10:55:21.940393 (+ 1964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:21.940399 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:21.940498 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:21.940633 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:21.941829 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:21.942581 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:21.944428 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:21.944955 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:21.945101 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:21.946706 (+ 1605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:21.946711 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:21.946714 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:21.946735 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:21.946778 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:21.946790 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:21.946904 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:21.947441 (+ 537us) server_negotiation.cc:299] Negotiation successful
0807 10:55:21.947556 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":359,"threads_started":1}
I20260807 10:55:21.949221 32080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:21.937177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:44099 (local address 127.3.73.194:46555)
0807 10:55:21.938212 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:21.938286 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:21.940070 (+ 1784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:21.940928 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:21.940937 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:21.941037 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:55:21.941640 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:21.941657 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:21.942769 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:21.942773 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:21.944266 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:21.944280 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:21.945213 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:21.945233 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:21.945483 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:21.946278 (+ 795us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 10:55:21.946281 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:55:21.946326 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 10:55:21.946441 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:55:21.946901 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:55:21.946916 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:55:21.946927 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 10:55:21.947428 (+ 501us) client_negotiation.cc:240] Negotiation successful
0807 10:55:21.947914 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":309,"threads_started":1}
I20260807 10:55:21.951064 32076 heartbeater.cc:347] Connected to a master server at 127.3.73.254:44099
I20260807 10:55:21.951440 32076 heartbeater.cc:464] Registering TS with master...
I20260807 10:55:21.952365 32076 heartbeater.cc:511] Master 127.3.73.254:44099 requested a full tablet report, sending...
I20260807 10:55:21.954385 31615 ts_manager.cc:194] Registered new tserver with Master: 54d7719a109c43588c7364ade59bb009 (127.3.73.194:42079)
I20260807 10:55:21.955681 31615 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.73.194:46555
W20260807 10:55:22.323397 32082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:55:22.323786 32082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:55:22.323964 32082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:55:22.331789 32082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:55:22.332010 32082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:55:22.332206 32082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.195
I20260807 10:55:22.342348 32082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:43693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:44099
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.195
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:55:22.343801 32082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:55:22.345759 32082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:55:22.361019 32092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:22.361063 32094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:22.361110 32091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:22.362861 32082 server_base.cc:1034] running on GCE node
I20260807 10:55:22.363626 32082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:55:22.364890 32082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:55:22.366170 32082 hybrid_clock.cc:648] HybridClock initialized: now 1786100122366117 us; error 45 us; skew 500 ppm
I20260807 10:55:22.366561 32082 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 10:55:22.375795 32082 webserver.cc:460] Webserver started at http://127.3.73.195:39529/ using document root <none> and password file <none>
I20260807 10:55:22.376832 32082 fs_manager.cc:359] Metadata directory not provided
I20260807 10:55:22.377041 32082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:55:22.377440 32082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:55:22.381062 32082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-2/data/instance:
uuid: "c26f1de1949f438b9852d828e26df811"
format_stamp: "Formatted at 2026-08-07 10:55:22 on dist-test-slave-x5fp"
server_key: "57ccbbf9205d62b1180d497cbb2931eb"
server_key_iv: "2c43909f5c5f896c6041cd0c482a7c3a"
server_key_version: "encryptionkey@0"
I20260807 10:55:22.382100 32082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance:
uuid: "c26f1de1949f438b9852d828e26df811"
format_stamp: "Formatted at 2026-08-07 10:55:22 on dist-test-slave-x5fp"
server_key: "57ccbbf9205d62b1180d497cbb2931eb"
server_key_iv: "2c43909f5c5f896c6041cd0c482a7c3a"
server_key_version: "encryptionkey@0"
I20260807 10:55:22.388556 32082 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 10:55:22.393126 32102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:22.394748 32082 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 10:55:22.395038 32082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-2/wal
uuid: "c26f1de1949f438b9852d828e26df811"
format_stamp: "Formatted at 2026-08-07 10:55:22 on dist-test-slave-x5fp"
server_key: "57ccbbf9205d62b1180d497cbb2931eb"
server_key_iv: "2c43909f5c5f896c6041cd0c482a7c3a"
server_key_version: "encryptionkey@0"
I20260807 10:55:22.395308 32082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:55:22.418632 32082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:55:22.419579 32082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:55:22.419924 32082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:55:22.421212 32082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:55:22.422916 32082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:55:22.423053 32082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:22.423185 32082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:55:22.423275 32082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:22.431625 31872 heartbeater.cc:503] Master 127.3.73.254:44099 was elected leader, sending a full tablet report...
I20260807 10:55:22.461571 32082 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.195:39983
I20260807 10:55:22.461669 32270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.195:39983 every 8 connection(s)
I20260807 10:55:22.463032 32082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
I20260807 10:55:22.469219 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 32082
I20260807 10:55:22.469421 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance
I20260807 10:55:22.469982 3367 external_mini_cluster.cc:1428] Setting key 7de691d30a77489b3227635691031bc1
I20260807 10:55:22.477810 32276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:22.466250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:33043 (local address 127.3.73.254:44099)
0807 10:55:22.466752 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:22.466757 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:22.468054 (+ 1297us) server_negotiation.cc:407] Connection header received
0807 10:55:22.470621 (+ 2567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:22.470627 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:22.470735 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:22.470873 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:22.472238 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:22.473123 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:22.474860 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:22.475263 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:22.475421 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:22.476860 (+ 1439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:22.476866 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:22.476871 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:22.476896 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:22.476946 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:22.476959 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:22.477067 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:22.477305 (+ 238us) server_negotiation.cc:299] Negotiation successful
0807 10:55:22.477418 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":242,"threads_started":1}
I20260807 10:55:22.478904 32277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:22.466864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:44099 (local address 127.3.73.195:33043)
0807 10:55:22.467865 (+ 1001us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:22.467912 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:22.470265 (+ 2353us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:22.471148 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:22.471160 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:22.471286 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:55:22.472056 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:22.472078 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:22.473300 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:22.473307 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:22.474702 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:22.474716 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:22.475521 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:22.475529 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:22.475775 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:22.476489 (+ 714us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 10:55:22.476491 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:55:22.476533 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 10:55:22.476622 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:55:22.477068 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:55:22.477082 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:55:22.477092 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 10:55:22.477293 (+ 201us) client_negotiation.cc:240] Negotiation successful
0807 10:55:22.477729 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":320,"threads_started":1}
I20260807 10:55:22.480427 32272 heartbeater.cc:347] Connected to a master server at 127.3.73.254:44099
I20260807 10:55:22.480854 32272 heartbeater.cc:464] Registering TS with master...
I20260807 10:55:22.481735 32272 heartbeater.cc:511] Master 127.3.73.254:44099 requested a full tablet report, sending...
I20260807 10:55:22.483332 31615 ts_manager.cc:194] Registered new tserver with Master: c26f1de1949f438b9852d828e26df811 (127.3.73.195:39983)
I20260807 10:55:22.484254 31615 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.73.195:33043
I20260807 10:55:22.486245 3367 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 10:55:22.497841 32276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:22.491129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58856 (local address 127.3.73.254:44099)
0807 10:55:22.491409 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:22.491420 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:22.491852 (+ 432us) server_negotiation.cc:407] Connection header received
0807 10:55:22.491975 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:22.491979 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:22.492052 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:22.492148 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:22.492678 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:22.493581 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:22.494959 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:22.495401 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:22.495506 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:22.496848 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:22.496855 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:22.496858 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:22.496881 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:22.496914 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:22.496930 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:22.497081 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:22.497325 (+ 244us) server_negotiation.cc:299] Negotiation successful
0807 10:55:22.497465 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 10:55:22.502964 31613 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58856:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 10:55:22.511739 31613 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 10:55:22.542290 32289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:22.527431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53546 (local address 127.3.73.195:39983)
0807 10:55:22.529153 (+ 1722us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:22.529166 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:22.531063 (+ 1897us) server_negotiation.cc:407] Connection header received
0807 10:55:22.531188 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:22.531201 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:22.531572 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:22.531894 (+ 322us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:55:22.533479 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:22.535362 (+ 1883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:22.539014 (+ 3652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:22.540578 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:22.540801 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:22.541165 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:22.541570 (+ 405us) server_negotiation.cc:299] Negotiation successful
0807 10:55:22.541832 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1508,"thread_start_us":1332,"threads_started":1}
I20260807 10:55:22.542352 32288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:22.526981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:39983 (local address 127.0.0.1:53546)
0807 10:55:22.529661 (+ 2680us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:22.529711 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:22.529950 (+ 239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:22.532344 (+ 2394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:22.532357 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:22.532406 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:55:22.533120 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:22.533142 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:22.535603 (+ 2461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:22.535612 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:22.538695 (+ 3083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:22.538712 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:22.540851 (+ 2139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:22.540858 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:22.541278 (+ 420us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:22.541332 (+ 54us) client_negotiation.cc:769] Sending connection context
0807 10:55:22.541510 (+ 178us) client_negotiation.cc:240] Negotiation successful
0807 10:55:22.541897 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":986,"mutex_wait_us":179,"thread_start_us":637,"threads_started":1}
I20260807 10:55:22.546411 32295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:22.530674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42988 (local address 127.3.73.193:33881)
0807 10:55:22.537897 (+ 7223us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:22.537905 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:22.537937 (+ 32us) server_negotiation.cc:407] Connection header received
0807 10:55:22.538051 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:22.538057 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:22.538303 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:22.538547 (+ 244us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:55:22.539553 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:22.540973 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:22.543554 (+ 2581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:22.545116 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:22.545314 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:22.545608 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:22.545840 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 10:55:22.546038 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6995,"thread_start_us":7091,"threads_started":1}
I20260807 10:55:22.546413 32294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:22.530229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:33881 (local address 127.0.0.1:42988)
0807 10:55:22.537153 (+ 6924us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:22.537175 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:22.537316 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:22.538832 (+ 1516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:22.538837 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:22.538864 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:55:22.539310 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:22.539323 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:22.541156 (+ 1833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:22.541162 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:22.543338 (+ 2176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:22.543350 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:22.545369 (+ 2019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:22.545377 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:22.545582 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:22.545675 (+ 93us) client_negotiation.cc:769] Sending connection context
0807 10:55:22.545788 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 10:55:22.545986 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6787,"mutex_wait_us":75,"thread_start_us":6620,"threads_started":1}
I20260807 10:55:22.546617 32291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:22.528080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59034 (local address 127.3.73.194:42079)
0807 10:55:22.536756 (+ 8676us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:22.536769 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:22.536811 (+ 42us) server_negotiation.cc:407] Connection header received
0807 10:55:22.536934 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:22.536944 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:22.537287 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:22.537546 (+ 259us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:55:22.538362 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:22.540154 (+ 1792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:22.543078 (+ 2924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:22.544795 (+ 1717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:22.544963 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:22.545246 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:22.545984 (+ 738us) server_negotiation.cc:299] Negotiation successful
0807 10:55:22.546210 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8466,"thread_start_us":8337,"threads_started":1}
I20260807 10:55:22.547160 32292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:22.527545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:42079 (local address 127.0.0.1:59034)
0807 10:55:22.530176 (+ 2631us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:22.530195 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:22.530305 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:22.537555 (+ 7250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:22.537561 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:22.537591 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:55:22.538093 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:22.538133 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:22.540327 (+ 2194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:22.540333 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:22.542822 (+ 2489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:22.542841 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:22.545431 (+ 2590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:22.545440 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:22.545688 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:22.545724 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 10:55:22.545824 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 10:55:22.546819 (+ 995us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1868,"mutex_wait_us":866,"thread_start_us":1555,"threads_started":1}
I20260807 10:55:22.550148 31789 tablet_service.cc:1467] Processing CreateTablet for tablet 8ac69fd4d2f04498828be83d71f38f09 (DEFAULT_TABLE table=test-table [id=6fb6698f49d740dab03c4ac915352a6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:55:22.547644 32176 tablet_service.cc:1467] Processing CreateTablet for tablet 8ac69fd4d2f04498828be83d71f38f09 (DEFAULT_TABLE table=test-table [id=6fb6698f49d740dab03c4ac915352a6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:55:22.550670 31972 tablet_service.cc:1467] Processing CreateTablet for tablet 8ac69fd4d2f04498828be83d71f38f09 (DEFAULT_TABLE table=test-table [id=6fb6698f49d740dab03c4ac915352a6b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:55:22.551862 31789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ac69fd4d2f04498828be83d71f38f09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:55:22.551862 32176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ac69fd4d2f04498828be83d71f38f09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:55:22.552083 31972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ac69fd4d2f04498828be83d71f38f09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:55:22.566601 32296 tablet_bootstrap.cc:492] T 8ac69fd4d2f04498828be83d71f38f09 P 54d7719a109c43588c7364ade59bb009: Bootstrap starting.
I20260807 10:55:22.569710 32300 tablet_bootstrap.cc:492] T 8ac69fd4d2f04498828be83d71f38f09 P 4b3d2d8691e24946820c8ccb3dbdcb8e: Bootstrap starting.
I20260807 10:55:22.570645 32301 tablet_bootstrap.cc:492] T 8ac69fd4d2f04498828be83d71f38f09 P c26f1de1949f438b9852d828e26df811: Bootstrap starting.
I20260807 10:55:22.572458 32296 tablet_bootstrap.cc:654] T 8ac69fd4d2f04498828be83d71f38f09 P 54d7719a109c43588c7364ade59bb009: Neither blocks nor log segments found. Creating new log.
I20260807 10:55:22.574043 32296 log.cc:824] T 8ac69fd4d2f04498828be83d71f38f09 P 54d7719a109c43588c7364ade59bb009: Log is configured to *not* fsync() on all Append() calls
I20260807 10:55:22.574292 32300 tablet_bootstrap.cc:654] T 8ac69fd4d2f04498828be83d71f38f09 P 4b3d2d8691e24946820c8ccb3dbdcb8e: Neither blocks nor log segments found. Creating new log.
I20260807 10:55:22.574759 32301 tablet_bootstrap.cc:654] T 8ac69fd4d2f04498828be83d71f38f09 P c26f1de1949f438b9852d828e26df811: Neither blocks nor log segments found. Creating new log.
I20260807 10:55:22.575619 32300 log.cc:824] T 8ac69fd4d2f04498828be83d71f38f09 P 4b3d2d8691e24946820c8ccb3dbdcb8e: Log is configured to *not* fsync() on all Append() calls
I20260807 10:55:22.575870 32301 log.cc:824] T 8ac69fd4d2f04498828be83d71f38f09 P c26f1de1949f438b9852d828e26df811: Log is configured to *not* fsync() on all Append() calls
I20260807 10:55:22.578101 32296 tablet_bootstrap.cc:492] T 8ac69fd4d2f04498828be83d71f38f09 P 54d7719a109c43588c7364ade59bb009: No bootstrap required, opened a new log
I20260807 10:55:22.578461 32296 ts_tablet_manager.cc:1397] T 8ac69fd4d2f04498828be83d71f38f09 P 54d7719a109c43588c7364ade59bb009: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260807 10:55:22.578922 32301 tablet_bootstrap.cc:492] T 8ac69fd4d2f04498828be83d71f38f09 P c26f1de1949f438b9852d828e26df811: No bootstrap required, opened a new log
I20260807 10:55:22.579130 32300 tablet_bootstrap.cc:492] T 8ac69fd4d2f04498828be83d71f38f09 P 4b3d2d8691e24946820c8ccb3dbdcb8e: No bootstrap required, opened a new log
I20260807 10:55:22.579190 32301 ts_tablet_manager.cc:1397] T 8ac69fd4d2f04498828be83d71f38f09 P c26f1de1949f438b9852d828e26df811: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260807 10:55:22.579429 32300 ts_tablet_manager.cc:1397] T 8ac69fd4d2f04498828be83d71f38f09 P 4b3d2d8691e24946820c8ccb3dbdcb8e: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
I20260807 10:55:22.586701 32301 raft_consensus.cc:348] T 8ac69fd4d2f04498828be83d71f38f09 P c26f1de1949f438b9852d828e26df811 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c26f1de1949f438b9852d828e26df811" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39983 } } peers { permanent_uuid: "54d7719a109c43588c7364ade59bb009" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 42079 } } peers { permanent_uuid: "4b3d2d8691e24946820c8ccb3dbdcb8e" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33881 } }
I20260807 10:55:22.587438 32301 raft_consensus.cc:374] T 8ac69fd4d2f04498828be83d71f38f09 P c26f1de1949f438b9852d828e26df811 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:55:22.587589 32301 raft_consensus.cc:729] T 8ac69fd4d2f04498828be83d71f38f09 P c26f1de1949f438b9852d828e26df811 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c26f1de1949f438b9852d828e26df811, State: Initialized, Role: FOLLOWER
I20260807 10:55:22.588433 32301 consensus_queue.cc:260] T 8ac69fd4d2f04498828be83d71f38f09 P c26f1de1949f438b9852d828e26df811 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c26f1de1949f438b9852d828e26df811" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39983 } } peers { permanent_uuid: "54d7719a109c43588c7364ade59bb009" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 42079 } } peers { permanent_uuid: "4b3d2d8691e24946820c8ccb3dbdcb8e" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33881 } }
I20260807 10:55:22.589828 32272 heartbeater.cc:503] Master 127.3.73.254:44099 was elected leader, sending a full tablet report...
I20260807 10:55:22.590638 32301 ts_tablet_manager.cc:1428] T 8ac69fd4d2f04498828be83d71f38f09 P c26f1de1949f438b9852d828e26df811: Time spent starting tablet: real 0.011s user 0.007s sys 0.005s
I20260807 10:55:22.591144 32296 raft_consensus.cc:348] T 8ac69fd4d2f04498828be83d71f38f09 P 54d7719a109c43588c7364ade59bb009 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c26f1de1949f438b9852d828e26df811" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39983 } } peers { permanent_uuid: "54d7719a109c43588c7364ade59bb009" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 42079 } } peers { permanent_uuid: "4b3d2d8691e24946820c8ccb3dbdcb8e" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33881 } }
I20260807 10:55:22.591205 32300 raft_consensus.cc:348] T 8ac69fd4d2f04498828be83d71f38f09 P 4b3d2d8691e24946820c8ccb3dbdcb8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c26f1de1949f438b9852d828e26df811" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39983 } } peers { permanent_uuid: "54d7719a109c43588c7364ade59bb009" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 42079 } } peers { permanent_uuid: "4b3d2d8691e24946820c8ccb3dbdcb8e" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33881 } }
I20260807 10:55:22.591672 32296 raft_consensus.cc:374] T 8ac69fd4d2f04498828be83d71f38f09 P 54d7719a109c43588c7364ade59bb009 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:55:22.591672 32300 raft_consensus.cc:374] T 8ac69fd4d2f04498828be83d71f38f09 P 4b3d2d8691e24946820c8ccb3dbdcb8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:55:22.591796 32300 raft_consensus.cc:729] T 8ac69fd4d2f04498828be83d71f38f09 P 4b3d2d8691e24946820c8ccb3dbdcb8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b3d2d8691e24946820c8ccb3dbdcb8e, State: Initialized, Role: FOLLOWER
I20260807 10:55:22.591795 32296 raft_consensus.cc:729] T 8ac69fd4d2f04498828be83d71f38f09 P 54d7719a109c43588c7364ade59bb009 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54d7719a109c43588c7364ade59bb009, State: Initialized, Role: FOLLOWER
I20260807 10:55:22.592567 32300 consensus_queue.cc:260] T 8ac69fd4d2f04498828be83d71f38f09 P 4b3d2d8691e24946820c8ccb3dbdcb8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c26f1de1949f438b9852d828e26df811" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39983 } } peers { permanent_uuid: "54d7719a109c43588c7364ade59bb009" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 42079 } } peers { permanent_uuid: "4b3d2d8691e24946820c8ccb3dbdcb8e" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33881 } }
I20260807 10:55:22.592566 32296 consensus_queue.cc:260] T 8ac69fd4d2f04498828be83d71f38f09 P 54d7719a109c43588c7364ade59bb009 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c26f1de1949f438b9852d828e26df811" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39983 } } peers { permanent_uuid: "54d7719a109c43588c7364ade59bb009" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 42079 } } peers { permanent_uuid: "4b3d2d8691e24946820c8ccb3dbdcb8e" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33881 } }
I20260807 10:55:22.594429 32076 heartbeater.cc:503] Master 127.3.73.254:44099 was elected leader, sending a full tablet report...
I20260807 10:55:22.595395 32296 ts_tablet_manager.cc:1428] T 8ac69fd4d2f04498828be83d71f38f09 P 54d7719a109c43588c7364ade59bb009: Time spent starting tablet: real 0.017s user 0.012s sys 0.004s
I20260807 10:55:22.597158 32300 ts_tablet_manager.cc:1428] T 8ac69fd4d2f04498828be83d71f38f09 P 4b3d2d8691e24946820c8ccb3dbdcb8e: Time spent starting tablet: real 0.018s user 0.015s sys 0.003s
W20260807 10:55:22.653934 31874 tablet.cc:2369] T 8ac69fd4d2f04498828be83d71f38f09 P 4b3d2d8691e24946820c8ccb3dbdcb8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 10:55:22.686048 32077 tablet.cc:2369] T 8ac69fd4d2f04498828be83d71f38f09 P 54d7719a109c43588c7364ade59bb009: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 10:55:22.715039 32274 tablet.cc:2369] T 8ac69fd4d2f04498828be83d71f38f09 P c26f1de1949f438b9852d828e26df811: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:55:22.750314 32307 raft_consensus.cc:482] T 8ac69fd4d2f04498828be83d71f38f09 P 54d7719a109c43588c7364ade59bb009 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 10:55:22.750658 32307 raft_consensus.cc:504] T 8ac69fd4d2f04498828be83d71f38f09 P 54d7719a109c43588c7364ade59bb009 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c26f1de1949f438b9852d828e26df811" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39983 } } peers { permanent_uuid: "54d7719a109c43588c7364ade59bb009" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 42079 } } peers { permanent_uuid: "4b3d2d8691e24946820c8ccb3dbdcb8e" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33881 } }
I20260807 10:55:22.752553 32307 leader_election.cc:288] T 8ac69fd4d2f04498828be83d71f38f09 P 54d7719a109c43588c7364ade59bb009 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c26f1de1949f438b9852d828e26df811 (127.3.73.195:39983), 4b3d2d8691e24946820c8ccb3dbdcb8e (127.3.73.193:33881)
I20260807 10:55:22.761379 32295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:22.754350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:53531 (local address 127.3.73.193:33881)
0807 10:55:22.754554 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:22.754560 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:22.755234 (+ 674us) server_negotiation.cc:407] Connection header received
0807 10:55:22.755449 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:22.755456 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:22.755548 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:22.755653 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:55:22.756247 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:22.757443 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:22.759575 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:22.760374 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:22.760570 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:22.760648 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:22.760999 (+ 351us) server_negotiation.cc:299] Negotiation successful
0807 10:55:22.761134 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 10:55:22.761682 32289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:22.753025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:55001 (local address 127.3.73.195:39983)
0807 10:55:22.753217 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:22.753222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:22.754123 (+ 901us) server_negotiation.cc:407] Connection header received
0807 10:55:22.754477 (+ 354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:22.754483 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:22.754572 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:22.754653 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:55:22.755581 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:22.756580 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:22.758963 (+ 2383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:22.760367 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:22.760549 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:22.760627 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:22.761255 (+ 628us) server_negotiation.cc:299] Negotiation successful
0807 10:55:22.761421 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 10:55:22.761725 32309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:22.754134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:33881 (local address 127.3.73.194:53531)
0807 10:55:22.755085 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:22.755107 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:22.755254 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:22.755685 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:22.755689 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:22.755719 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:55:22.756075 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:22.756088 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:22.757599 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:22.757605 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:22.759381 (+ 1776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:22.759392 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:22.760574 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:22.760580 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:22.760825 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:22.760852 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 10:55:22.760967 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 10:55:22.761103 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":809,"mutex_wait_us":5,"thread_start_us":585,"threads_started":1}
I20260807 10:55:22.762418 32308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:22.753211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:39983 (local address 127.3.73.194:55001)
0807 10:55:22.753951 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:22.753969 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:22.754269 (+ 300us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:22.754914 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:22.754918 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:22.754943 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:55:22.755423 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:22.755442 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:22.756714 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:22.756746 (+ 32us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:22.758733 (+ 1987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:22.758749 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:22.760812 (+ 2063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:22.760827 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:22.761053 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:22.761091 (+ 38us) client_negotiation.cc:769] Sending connection context
0807 10:55:22.761882 (+ 791us) client_negotiation.cc:240] Negotiation successful
0807 10:55:22.762046 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"mutex_wait_us":55,"thread_start_us":394,"threads_started":1}
I20260807 10:55:22.762939 31814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ac69fd4d2f04498828be83d71f38f09" candidate_uuid: "54d7719a109c43588c7364ade59bb009" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3d2d8691e24946820c8ccb3dbdcb8e" is_pre_election: true
I20260807 10:55:22.763550 31814 raft_consensus.cc:2454] T 8ac69fd4d2f04498828be83d71f38f09 P 4b3d2d8691e24946820c8ccb3dbdcb8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54d7719a109c43588c7364ade59bb009 in term 0.
I20260807 10:55:22.764276 31902 leader_election.cc:302] T 8ac69fd4d2f04498828be83d71f38f09 P 54d7719a109c43588c7364ade59bb009 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b3d2d8691e24946820c8ccb3dbdcb8e, 54d7719a109c43588c7364ade59bb009; no voters:
I20260807 10:55:22.764354 32202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ac69fd4d2f04498828be83d71f38f09" candidate_uuid: "54d7719a109c43588c7364ade59bb009" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c26f1de1949f438b9852d828e26df811" is_pre_election: true
I20260807 10:55:22.764711 32307 raft_consensus.cc:2781] T 8ac69fd4d2f04498828be83d71f38f09 P 54d7719a109c43588c7364ade59bb009 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 10:55:22.764896 32307 raft_consensus.cc:482] T 8ac69fd4d2f04498828be83d71f38f09 P 54d7719a109c43588c7364ade59bb009 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 10:55:22.764969 32307 raft_consensus.cc:3037] T 8ac69fd4d2f04498828be83d71f38f09 P 54d7719a109c43588c7364ade59bb009 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:55:22.765023 32202 raft_consensus.cc:2454] T 8ac69fd4d2f04498828be83d71f38f09 P c26f1de1949f438b9852d828e26df811 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54d7719a109c43588c7364ade59bb009 in term 0.
I20260807 10:55:22.767341 32307 raft_consensus.cc:504] T 8ac69fd4d2f04498828be83d71f38f09 P 54d7719a109c43588c7364ade59bb009 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c26f1de1949f438b9852d828e26df811" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39983 } } peers { permanent_uuid: "54d7719a109c43588c7364ade59bb009" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 42079 } } peers { permanent_uuid: "4b3d2d8691e24946820c8ccb3dbdcb8e" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33881 } }
I20260807 10:55:22.768009 32307 leader_election.cc:288] T 8ac69fd4d2f04498828be83d71f38f09 P 54d7719a109c43588c7364ade59bb009 [CANDIDATE]: Term 1 election: Requested vote from peers c26f1de1949f438b9852d828e26df811 (127.3.73.195:39983), 4b3d2d8691e24946820c8ccb3dbdcb8e (127.3.73.193:33881)
I20260807 10:55:22.768546 31814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ac69fd4d2f04498828be83d71f38f09" candidate_uuid: "54d7719a109c43588c7364ade59bb009" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b3d2d8691e24946820c8ccb3dbdcb8e"
I20260807 10:55:22.768546 32202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ac69fd4d2f04498828be83d71f38f09" candidate_uuid: "54d7719a109c43588c7364ade59bb009" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c26f1de1949f438b9852d828e26df811"
I20260807 10:55:22.768795 32202 raft_consensus.cc:3037] T 8ac69fd4d2f04498828be83d71f38f09 P c26f1de1949f438b9852d828e26df811 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:55:22.768827 31814 raft_consensus.cc:3037] T 8ac69fd4d2f04498828be83d71f38f09 P 4b3d2d8691e24946820c8ccb3dbdcb8e [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:55:22.772202 31814 raft_consensus.cc:2454] T 8ac69fd4d2f04498828be83d71f38f09 P 4b3d2d8691e24946820c8ccb3dbdcb8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54d7719a109c43588c7364ade59bb009 in term 1.
I20260807 10:55:22.772202 32202 raft_consensus.cc:2454] T 8ac69fd4d2f04498828be83d71f38f09 P c26f1de1949f438b9852d828e26df811 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54d7719a109c43588c7364ade59bb009 in term 1.
I20260807 10:55:22.772644 31902 leader_election.cc:302] T 8ac69fd4d2f04498828be83d71f38f09 P 54d7719a109c43588c7364ade59bb009 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54d7719a109c43588c7364ade59bb009, c26f1de1949f438b9852d828e26df811; no voters:
I20260807 10:55:22.773124 32307 raft_consensus.cc:2781] T 8ac69fd4d2f04498828be83d71f38f09 P 54d7719a109c43588c7364ade59bb009 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:55:22.773702 32307 raft_consensus.cc:686] T 8ac69fd4d2f04498828be83d71f38f09 P 54d7719a109c43588c7364ade59bb009 [term 1 LEADER]: Becoming Leader. State: Replica: 54d7719a109c43588c7364ade59bb009, State: Running, Role: LEADER
I20260807 10:55:22.774267 32307 consensus_queue.cc:237] T 8ac69fd4d2f04498828be83d71f38f09 P 54d7719a109c43588c7364ade59bb009 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c26f1de1949f438b9852d828e26df811" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39983 } } peers { permanent_uuid: "54d7719a109c43588c7364ade59bb009" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 42079 } } peers { permanent_uuid: "4b3d2d8691e24946820c8ccb3dbdcb8e" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33881 } }
I20260807 10:55:22.779523 31613 catalog_manager.cc:5495] T 8ac69fd4d2f04498828be83d71f38f09 P 54d7719a109c43588c7364ade59bb009 reported cstate change: term changed from 0 to 1, leader changed from <none> to 54d7719a109c43588c7364ade59bb009 (127.3.73.194). New cstate: current_term: 1 leader_uuid: "54d7719a109c43588c7364ade59bb009" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c26f1de1949f438b9852d828e26df811" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54d7719a109c43588c7364ade59bb009" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 42079 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b3d2d8691e24946820c8ccb3dbdcb8e" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33881 } health_report { overall_health: UNKNOWN } } }
I20260807 10:55:22.840341 32291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:22.834894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59036 (local address 127.3.73.194:42079)
0807 10:55:22.835105 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:22.835110 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:22.835131 (+ 21us) server_negotiation.cc:407] Connection header received
0807 10:55:22.835229 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:22.835233 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:22.835330 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:22.835464 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:55:22.835884 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:22.836591 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:22.838080 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:22.838507 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:22.838607 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:22.839132 (+ 525us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:55:22.839627 (+ 495us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:55:22.839767 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:22.839944 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 10:55:22.840012 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 10:55:22.849507 31814 raft_consensus.cc:1261] T 8ac69fd4d2f04498828be83d71f38f09 P 4b3d2d8691e24946820c8ccb3dbdcb8e [term 1 FOLLOWER]: Refusing update from remote peer 54d7719a109c43588c7364ade59bb009: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:55:22.849507 32202 raft_consensus.cc:1261] T 8ac69fd4d2f04498828be83d71f38f09 P c26f1de1949f438b9852d828e26df811 [term 1 FOLLOWER]: Refusing update from remote peer 54d7719a109c43588c7364ade59bb009: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:55:22.850745 32311 consensus_queue.cc:1035] T 8ac69fd4d2f04498828be83d71f38f09 P 54d7719a109c43588c7364ade59bb009 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c26f1de1949f438b9852d828e26df811" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 39983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:55:22.851222 32307 consensus_queue.cc:1035] T 8ac69fd4d2f04498828be83d71f38f09 P 54d7719a109c43588c7364ade59bb009 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b3d2d8691e24946820c8ccb3dbdcb8e" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:55:22.865163 32314 mvcc.cc:205] Tried to move back new op lower bound from 7315866103179284480 to 7315866102888517632. Current Snapshot: MvccSnapshot[applied={T|T < 7315866103179284480}]
I20260807 10:55:22.880347 31613 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:55:22.880645 31613 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58856:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:55:22.885150 31613 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 17e9fb6923b24fce9666abdcfbf1c30b: Sending DeleteTablet for 3 replicas of tablet 8ac69fd4d2f04498828be83d71f38f09
I20260807 10:55:22.886756 32176 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ac69fd4d2f04498828be83d71f38f09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:55:22 UTC) from {username='slave'} at 127.0.0.1:53546
I20260807 10:55:22.886849 31972 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ac69fd4d2f04498828be83d71f38f09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:55:22 UTC) from {username='slave'} at 127.0.0.1:59034
I20260807 10:55:22.887936 31789 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ac69fd4d2f04498828be83d71f38f09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:55:22 UTC) from {username='slave'} at 127.0.0.1:42988
I20260807 10:55:22.891620 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 31704
I20260807 10:55:22.907554 32329 tablet_replica.cc:330] T 8ac69fd4d2f04498828be83d71f38f09 P 4b3d2d8691e24946820c8ccb3dbdcb8e: stopping tablet replica
I20260807 10:55:22.908222 32329 raft_consensus.cc:2229] T 8ac69fd4d2f04498828be83d71f38f09 P 4b3d2d8691e24946820c8ccb3dbdcb8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:55:22.909025 32329 raft_consensus.cc:2258] T 8ac69fd4d2f04498828be83d71f38f09 P 4b3d2d8691e24946820c8ccb3dbdcb8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:55:22.915555 32329 ts_tablet_manager.cc:1905] T 8ac69fd4d2f04498828be83d71f38f09 P 4b3d2d8691e24946820c8ccb3dbdcb8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:55:22.915875 32327 tablet_replica.cc:330] T 8ac69fd4d2f04498828be83d71f38f09 P c26f1de1949f438b9852d828e26df811: stopping tablet replica
I20260807 10:55:22.916626 32327 raft_consensus.cc:2229] T 8ac69fd4d2f04498828be83d71f38f09 P c26f1de1949f438b9852d828e26df811 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:55:22.917482 32327 raft_consensus.cc:2258] T 8ac69fd4d2f04498828be83d71f38f09 P c26f1de1949f438b9852d828e26df811 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:55:22.923748 32329 ts_tablet_manager.cc:1918] T 8ac69fd4d2f04498828be83d71f38f09 P 4b3d2d8691e24946820c8ccb3dbdcb8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:55:22.924047 32329 log.cc:1196] T 8ac69fd4d2f04498828be83d71f38f09 P 4b3d2d8691e24946820c8ccb3dbdcb8e: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-0/wal/wals/8ac69fd4d2f04498828be83d71f38f09
I20260807 10:55:22.924410 32327 ts_tablet_manager.cc:1905] T 8ac69fd4d2f04498828be83d71f38f09 P c26f1de1949f438b9852d828e26df811: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:55:22.924666 32329 ts_tablet_manager.cc:1939] T 8ac69fd4d2f04498828be83d71f38f09 P 4b3d2d8691e24946820c8ccb3dbdcb8e: Deleting consensus metadata
I20260807 10:55:22.926927 31596 catalog_manager.cc:4841] TS 4b3d2d8691e24946820c8ccb3dbdcb8e (127.3.73.193:33881): tablet 8ac69fd4d2f04498828be83d71f38f09 (table test-table [id=6fb6698f49d740dab03c4ac915352a6b]) successfully deleted
I20260807 10:55:22.928447 32328 tablet_replica.cc:330] T 8ac69fd4d2f04498828be83d71f38f09 P 54d7719a109c43588c7364ade59bb009: stopping tablet replica
I20260807 10:55:22.929142 32328 raft_consensus.cc:2229] T 8ac69fd4d2f04498828be83d71f38f09 P 54d7719a109c43588c7364ade59bb009 [term 1 LEADER]: Raft consensus shutting down.
I20260807 10:55:22.929924 32328 raft_consensus.cc:2258] T 8ac69fd4d2f04498828be83d71f38f09 P 54d7719a109c43588c7364ade59bb009 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:55:22.930442 32295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:22.893924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43002 (local address 127.3.73.193:33881)
0807 10:55:22.894241 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:22.894253 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:22.907645 (+ 13392us) server_negotiation.cc:407] Connection header received
0807 10:55:22.908417 (+ 772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:22.908429 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:22.908665 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:22.908862 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:22.909985 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:22.911827 (+ 1842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:22.918191 (+ 6364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:22.919492 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:22.919672 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:22.928042 (+ 8370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:22.928058 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:22.928080 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:22.929178 (+ 1098us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:22.929261 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:22.929303 (+ 42us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:22.929478 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:22.929792 (+ 314us) server_negotiation.cc:299] Negotiation successful
0807 10:55:22.929951 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260807 10:55:22.932076 31866 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:55:22.932530 32328 ts_tablet_manager.cc:1905] T 8ac69fd4d2f04498828be83d71f38f09 P 54d7719a109c43588c7364ade59bb009: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:55:22.937309 32327 ts_tablet_manager.cc:1918] T 8ac69fd4d2f04498828be83d71f38f09 P c26f1de1949f438b9852d828e26df811: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:55:22.937752 32327 log.cc:1196] T 8ac69fd4d2f04498828be83d71f38f09 P c26f1de1949f438b9852d828e26df811: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-2/wal/wals/8ac69fd4d2f04498828be83d71f38f09
I20260807 10:55:22.938689 32327 ts_tablet_manager.cc:1939] T 8ac69fd4d2f04498828be83d71f38f09 P c26f1de1949f438b9852d828e26df811: Deleting consensus metadata
I20260807 10:55:22.940189 32328 ts_tablet_manager.cc:1918] T 8ac69fd4d2f04498828be83d71f38f09 P 54d7719a109c43588c7364ade59bb009: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:55:22.940354 32328 log.cc:1196] T 8ac69fd4d2f04498828be83d71f38f09 P 54d7719a109c43588c7364ade59bb009: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786099767749326-3367-0/minicluster-data/ts-1/wal/wals/8ac69fd4d2f04498828be83d71f38f09
I20260807 10:55:22.940905 32328 ts_tablet_manager.cc:1939] T 8ac69fd4d2f04498828be83d71f38f09 P 54d7719a109c43588c7364ade59bb009: Deleting consensus metadata
I20260807 10:55:22.941987 31596 catalog_manager.cc:4841] TS c26f1de1949f438b9852d828e26df811 (127.3.73.195:39983): tablet 8ac69fd4d2f04498828be83d71f38f09 (table test-table [id=6fb6698f49d740dab03c4ac915352a6b]) successfully deleted
I20260807 10:55:22.943166 31597 catalog_manager.cc:4841] TS 54d7719a109c43588c7364ade59bb009 (127.3.73.194:42079): tablet 8ac69fd4d2f04498828be83d71f38f09 (table test-table [id=6fb6698f49d740dab03c4ac915352a6b]) successfully deleted
I20260807 10:55:23.068070 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 31704
I20260807 10:55:23.104573 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 31878
I20260807 10:55:23.113438 32291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:23.105562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59044 (local address 127.3.73.194:42079)
0807 10:55:23.105834 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:23.105840 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:23.105866 (+ 26us) server_negotiation.cc:407] Connection header received
0807 10:55:23.105946 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:23.105951 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:23.106057 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:23.106202 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:23.106678 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:23.107946 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:23.109429 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:23.110148 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:23.110288 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:23.111512 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:23.111523 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:23.111544 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:23.112453 (+ 909us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:23.112522 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:23.112549 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:23.112679 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:23.112896 (+ 217us) server_negotiation.cc:299] Negotiation successful
0807 10:55:23.113031 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260807 10:55:23.114250 32071 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:55:23.233747 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 31878
I20260807 10:55:23.261226 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 32082
I20260807 10:55:23.268987 32336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:23.262080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53556 (local address 127.3.73.195:39983)
0807 10:55:23.262671 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:23.262676 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:23.262693 (+ 17us) server_negotiation.cc:407] Connection header received
0807 10:55:23.262752 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:23.262756 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:23.262837 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:23.262945 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:23.263380 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:23.264123 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:23.265669 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:23.266076 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:23.266184 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:23.267376 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:23.267385 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:23.267401 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:23.268052 (+ 651us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:23.268103 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:23.268125 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:23.268234 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:23.268423 (+ 189us) server_negotiation.cc:299] Negotiation successful
0807 10:55:23.268528 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":346,"threads_started":1}
I20260807 10:55:23.269784 32262 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:55:23.362144 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 32082
I20260807 10:55:23.390064 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 31573
I20260807 10:55:23.391350 31663 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:55:23.507076 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 31573
2026-08-07T10:55:23Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3395 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:55:23 dist-test-slave-x5fp krb5kdc[32343](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:55:23 dist-test-slave-x5fp krb5kdc[32343](info): set up 2 sockets
Aug 07 10:55:23 dist-test-slave-x5fp krb5kdc[32343](info): commencing operation
krb5kdc: starting...
W20260807 10:55:24.899048 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.338s user 0.001s sys 0.058s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:55:25 dist-test-slave-x5fp krb5kdc[32343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:55:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:55:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.73.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
I20260807 10:55:25.284813 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:41493
--webserver_interface=127.3.73.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:39973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:41493
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:55:25.661608 32363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:55:25.661960 32363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:55:25.662156 32363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:55:25.670049 32363 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:55:25.670176 32363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:55:25.670295 32363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:55:25.670397 32363 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:55:25.670483 32363 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:55:25.682484 32363 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:39973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:41493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:41493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:55:25.684083 32363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:55:25.685927 32363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:55:25.699172 32371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:25.699285 32373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:25.699245 32370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:25.700274 32363 server_base.cc:1034] running on GCE node
I20260807 10:55:25.701264 32363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:55:25.702849 32363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:55:25.704097 32363 hybrid_clock.cc:648] HybridClock initialized: now 1786100125704019 us; error 55 us; skew 500 ppm
I20260807 10:55:25.704535 32363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:55:25 dist-test-slave-x5fp krb5kdc[32343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100125, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:55:25.708050 32363 init.cc:377] Logged in from keytab as kudu/127.3.73.254@KRBTEST.COM (short username kudu)
I20260807 10:55:25.709887 32363 webserver.cc:460] Webserver started at http://127.3.73.254:44703/ using document root <none> and password file <none>
I20260807 10:55:25.710655 32363 fs_manager.cc:359] Metadata directory not provided
I20260807 10:55:25.710815 32363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:55:25.711195 32363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:55:25.714571 32363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "7d4a94a408064cb3ba526d92dbd9d089"
format_stamp: "Formatted at 2026-08-07 10:55:25 on dist-test-slave-x5fp"
server_key: "ac59090cdc6bf1de7e337d5d812222f7"
server_key_iv: "3c1dcdb13d4a9881ea15c660fa84058a"
server_key_version: "encryptionkey@0"
I20260807 10:55:25.715435 32363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "7d4a94a408064cb3ba526d92dbd9d089"
format_stamp: "Formatted at 2026-08-07 10:55:25 on dist-test-slave-x5fp"
server_key: "ac59090cdc6bf1de7e337d5d812222f7"
server_key_iv: "3c1dcdb13d4a9881ea15c660fa84058a"
server_key_version: "encryptionkey@0"
I20260807 10:55:25.721467 32363 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260807 10:55:25.725543 32382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:25.727365 32363 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260807 10:55:25.727582 32363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "7d4a94a408064cb3ba526d92dbd9d089"
format_stamp: "Formatted at 2026-08-07 10:55:25 on dist-test-slave-x5fp"
server_key: "ac59090cdc6bf1de7e337d5d812222f7"
server_key_iv: "3c1dcdb13d4a9881ea15c660fa84058a"
server_key_version: "encryptionkey@0"
I20260807 10:55:25.727841 32363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:55:25.755038 32363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:55:25.760042 32363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:55:25.760401 32363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:55:25.781323 32363 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:41493
I20260807 10:55:25.781383 32460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:41493 every 8 connection(s)
I20260807 10:55:25.782842 32363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:55:25.784035 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 32363
I20260807 10:55:25.784257 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:55:25.784894 3367 external_mini_cluster.cc:1428] Setting key 86732326f641dbf454195777ab0808dd
I20260807 10:55:25.788110 32461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 10:55:25 dist-test-slave-x5fp krb5kdc[32343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:55:25.802639 32461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d4a94a408064cb3ba526d92dbd9d089: Bootstrap starting.
I20260807 10:55:25.807446 32461 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d4a94a408064cb3ba526d92dbd9d089: Neither blocks nor log segments found. Creating new log.
I20260807 10:55:25.808831 32461 log.cc:824] T 00000000000000000000000000000000 P 7d4a94a408064cb3ba526d92dbd9d089: Log is configured to *not* fsync() on all Append() calls
I20260807 10:55:25.812657 32461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d4a94a408064cb3ba526d92dbd9d089: No bootstrap required, opened a new log
I20260807 10:55:25.813180 32464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:25.787040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46752 (local address 127.3.73.254:41493)
0807 10:55:25.787916 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:25.787928 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:25.787970 (+ 42us) server_negotiation.cc:407] Connection header received
0807 10:55:25.789851 (+ 1881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:25.789876 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:25.790689 (+ 813us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:25.791394 (+ 705us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:25.793401 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:25.794920 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:25.796858 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:25.797593 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:25.797747 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:25.803299 (+ 5552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:25.803311 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:25.803335 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:55:25.803398 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:25.807849 (+ 4451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:25.808805 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:25.808821 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:25.808830 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:25.809052 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:25.809543 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:25.809552 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:25.809555 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:25.810240 (+ 685us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:25.810580 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:25.811041 (+ 461us) server_negotiation.cc:299] Negotiation successful
0807 10:55:25.811531 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":418,"threads_started":1}
I20260807 10:55:25.822687 32461 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d4a94a408064cb3ba526d92dbd9d089 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4a94a408064cb3ba526d92dbd9d089" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41493 } }
I20260807 10:55:25.823149 32461 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d4a94a408064cb3ba526d92dbd9d089 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:55:25.823246 32461 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d4a94a408064cb3ba526d92dbd9d089 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d4a94a408064cb3ba526d92dbd9d089, State: Initialized, Role: FOLLOWER
I20260807 10:55:25.823841 32461 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d4a94a408064cb3ba526d92dbd9d089 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4a94a408064cb3ba526d92dbd9d089" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41493 } }
I20260807 10:55:25.824070 32461 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d4a94a408064cb3ba526d92dbd9d089 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:55:25.824182 32461 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d4a94a408064cb3ba526d92dbd9d089 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:55:25.824360 32461 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d4a94a408064cb3ba526d92dbd9d089 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:55:25.827365 32461 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d4a94a408064cb3ba526d92dbd9d089 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4a94a408064cb3ba526d92dbd9d089" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41493 } }
I20260807 10:55:25.827966 32461 leader_election.cc:302] T 00000000000000000000000000000000 P 7d4a94a408064cb3ba526d92dbd9d089 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d4a94a408064cb3ba526d92dbd9d089; no voters:
I20260807 10:55:25.828636 32461 leader_election.cc:288] T 00000000000000000000000000000000 P 7d4a94a408064cb3ba526d92dbd9d089 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:55:25.828882 32467 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d4a94a408064cb3ba526d92dbd9d089 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:55:25.829701 32467 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d4a94a408064cb3ba526d92dbd9d089 [term 1 LEADER]: Becoming Leader. State: Replica: 7d4a94a408064cb3ba526d92dbd9d089, State: Running, Role: LEADER
I20260807 10:55:25.830314 32467 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d4a94a408064cb3ba526d92dbd9d089 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4a94a408064cb3ba526d92dbd9d089" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41493 } }
I20260807 10:55:25.831205 32461 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d4a94a408064cb3ba526d92dbd9d089 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:55:25.834944 32468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d4a94a408064cb3ba526d92dbd9d089 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d4a94a408064cb3ba526d92dbd9d089" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4a94a408064cb3ba526d92dbd9d089" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41493 } } }
I20260807 10:55:25.834921 32469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d4a94a408064cb3ba526d92dbd9d089 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d4a94a408064cb3ba526d92dbd9d089. Latest consensus state: current_term: 1 leader_uuid: "7d4a94a408064cb3ba526d92dbd9d089" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d4a94a408064cb3ba526d92dbd9d089" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41493 } } }
I20260807 10:55:25.835405 32468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d4a94a408064cb3ba526d92dbd9d089 [sys.catalog]: This master's current role is: LEADER
I20260807 10:55:25.835405 32469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d4a94a408064cb3ba526d92dbd9d089 [sys.catalog]: This master's current role is: LEADER
I20260807 10:55:25.836879 32479 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:55:25.842048 32479 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:55:25.852186 32479 catalog_manager.cc:1347] Generated new cluster ID: 357075befb1d48e8b8773b1a952caffe
I20260807 10:55:25.852334 32479 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:55:25.872181 32479 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:55:25.874485 32479 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:55:25.883172 32479 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d4a94a408064cb3ba526d92dbd9d089: Generated new TSK 0
I20260807 10:55:25.884513 32479 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.73.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab.
Entry for principal kudu/127.3.73.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab.
I20260807 10:55:25.980564 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.193:0
--local_ip_for_outbound_sockets=127.3.73.193
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:41493
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:39973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:55:26.376065 32498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:55:26.376433 32498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:55:26.376631 32498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:55:26.385190 32498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:55:26.385357 32498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:55:26.385601 32498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.193
I20260807 10:55:26.397753 32498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:39973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.193.keytab
--principal=kudu/127.3.73.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:41493
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.193
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:55:26.399293 32498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:55:26.400802 32498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:55:26.415778 32506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:26.415876 32509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:26.415805 32504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:26.416827 32498 server_base.cc:1034] running on GCE node
I20260807 10:55:26.417563 32498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:55:26.418728 32498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:55:26.419996 32498 hybrid_clock.cc:648] HybridClock initialized: now 1786100126419939 us; error 49 us; skew 500 ppm
I20260807 10:55:26.420352 32498 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:55:26 dist-test-slave-x5fp krb5kdc[32343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100126, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:55:26.423956 32498 init.cc:377] Logged in from keytab as kudu/127.3.73.193@KRBTEST.COM (short username kudu)
I20260807 10:55:26.425756 32498 webserver.cc:460] Webserver started at http://127.3.73.193:41553/ using document root <none> and password file <none>
I20260807 10:55:26.426506 32498 fs_manager.cc:359] Metadata directory not provided
I20260807 10:55:26.426667 32498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:55:26.427035 32498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:55:26.430378 32498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-0/data/instance:
uuid: "191f08df78124986acbc92fc3ab9d2d3"
format_stamp: "Formatted at 2026-08-07 10:55:26 on dist-test-slave-x5fp"
server_key: "d687264d694d5e661b465aa3aaeea658"
server_key_iv: "8c9e88c85cc07dc33cb969d12364ef9f"
server_key_version: "encryptionkey@0"
I20260807 10:55:26.431320 32498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance:
uuid: "191f08df78124986acbc92fc3ab9d2d3"
format_stamp: "Formatted at 2026-08-07 10:55:26 on dist-test-slave-x5fp"
server_key: "d687264d694d5e661b465aa3aaeea658"
server_key_iv: "8c9e88c85cc07dc33cb969d12364ef9f"
server_key_version: "encryptionkey@0"
I20260807 10:55:26.437538 32498 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 10:55:26.441881 32519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:26.443437 32498 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.003s
I20260807 10:55:26.443650 32498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-0/wal
uuid: "191f08df78124986acbc92fc3ab9d2d3"
format_stamp: "Formatted at 2026-08-07 10:55:26 on dist-test-slave-x5fp"
server_key: "d687264d694d5e661b465aa3aaeea658"
server_key_iv: "8c9e88c85cc07dc33cb969d12364ef9f"
server_key_version: "encryptionkey@0"
I20260807 10:55:26.443912 32498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:55:26.462263 32498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:55:26.467196 32498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:55:26.467543 32498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:55:26.468648 32498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:55:26.470283 32498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:55:26.470393 32498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:26.470573 32498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:55:26.470649 32498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:26.508275 32498 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.193:33757
I20260807 10:55:26.508368 32674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.193:33757 every 8 connection(s)
I20260807 10:55:26.509702 32498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
I20260807 10:55:26.513200 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 32498
I20260807 10:55:26.513422 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance
I20260807 10:55:26.514020 3367 external_mini_cluster.cc:1428] Setting key fcad0c674367744c316c708980c48c72
Aug 07 10:55:26 dist-test-slave-x5fp krb5kdc[32343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100126, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.193@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:55:26.532648 32678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:26.512440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:43241 (local address 127.3.73.254:41493)
0807 10:55:26.512971 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:26.512976 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:26.514116 (+ 1140us) server_negotiation.cc:407] Connection header received
0807 10:55:26.516353 (+ 2237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:26.516358 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:26.516457 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:26.516589 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:26.518312 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:26.519041 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:26.520744 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:26.521331 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:26.521509 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:26.525512 (+ 4003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:26.525517 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:26.525520 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:55:26.525556 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:26.528861 (+ 3305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:26.529751 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:26.529756 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:26.529760 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:26.529905 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:26.530391 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:26.530395 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:26.530397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:26.530711 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:26.530877 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:26.531947 (+ 1070us) server_negotiation.cc:299] Negotiation successful
0807 10:55:26.532279 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":331,"threads_started":1}
I20260807 10:55:26.534112 32679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:26.513022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41493 (local address 127.3.73.193:43241)
0807 10:55:26.513946 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:26.513987 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:26.516072 (+ 2085us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:26.516926 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:26.516936 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:26.517511 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:55:26.518125 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:26.518144 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:26.519229 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:26.519233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:26.520555 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:26.520570 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:26.521636 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:26.521644 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:26.521927 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:26.522712 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:55:26.522742 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:55:26.525228 (+ 2486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:55:26.529070 (+ 3842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:55:26.529084 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:55:26.529106 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:55:26.529605 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:55:26.530029 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:55:26.530034 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:55:26.530038 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:55:26.530246 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:55:26.530858 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:55:26.530868 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:55:26.531395 (+ 527us) client_negotiation.cc:769] Sending connection context
0807 10:55:26.531789 (+ 394us) client_negotiation.cc:240] Negotiation successful
0807 10:55:26.532486 (+ 697us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":427,"threads_started":1}
I20260807 10:55:26.535897 32675 heartbeater.cc:347] Connected to a master server at 127.3.73.254:41493
I20260807 10:55:26.536293 32675 heartbeater.cc:464] Registering TS with master...
I20260807 10:55:26.537279 32675 heartbeater.cc:511] Master 127.3.73.254:41493 requested a full tablet report, sending...
I20260807 10:55:26.539920 32409 ts_manager.cc:194] Registered new tserver with Master: 191f08df78124986acbc92fc3ab9d2d3 (127.3.73.193:33757)
I20260807 10:55:26.541608 32409 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.193@KRBTEST.COM'} at 127.3.73.193:43241
WARNING: no policy specified for kudu/127.3.73.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab.
Entry for principal kudu/127.3.73.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab.
I20260807 10:55:26.598940 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.194:0
--local_ip_for_outbound_sockets=127.3.73.194
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:41493
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:39973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:55:26.980751 32682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:55:26.981084 32682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:55:26.981273 32682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:55:26.989027 32682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:55:26.989162 32682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:55:26.989358 32682 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.194
I20260807 10:55:26.999737 32682 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:39973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.194.keytab
--principal=kudu/127.3.73.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:41493
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.194
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:55:27.001308 32682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:55:27.002795 32682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:55:27.016909 32688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:27.017139 32689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:27.016940 32692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:27.018388 32682 server_base.cc:1034] running on GCE node
I20260807 10:55:27.019152 32682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:55:27.020211 32682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:55:27.021478 32682 hybrid_clock.cc:648] HybridClock initialized: now 1786100127021427 us; error 59 us; skew 500 ppm
I20260807 10:55:27.021872 32682 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:55:27 dist-test-slave-x5fp krb5kdc[32343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100127, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:55:27.025529 32682 init.cc:377] Logged in from keytab as kudu/127.3.73.194@KRBTEST.COM (short username kudu)
I20260807 10:55:27.027415 32682 webserver.cc:460] Webserver started at http://127.3.73.194:35357/ using document root <none> and password file <none>
I20260807 10:55:27.028198 32682 fs_manager.cc:359] Metadata directory not provided
I20260807 10:55:27.028357 32682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:55:27.028712 32682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:55:27.032254 32682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-1/data/instance:
uuid: "d5aa5495c77746a1b0c822b8c98f9081"
format_stamp: "Formatted at 2026-08-07 10:55:27 on dist-test-slave-x5fp"
server_key: "07f3e12d45749392ed9fcb2ec5e1d6c5"
server_key_iv: "194c8909bc4a39372a24713c335f099d"
server_key_version: "encryptionkey@0"
I20260807 10:55:27.033242 32682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance:
uuid: "d5aa5495c77746a1b0c822b8c98f9081"
format_stamp: "Formatted at 2026-08-07 10:55:27 on dist-test-slave-x5fp"
server_key: "07f3e12d45749392ed9fcb2ec5e1d6c5"
server_key_iv: "194c8909bc4a39372a24713c335f099d"
server_key_version: "encryptionkey@0"
I20260807 10:55:27.039367 32682 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 10:55:27.043356 32705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:27.044955 32682 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 10:55:27.045158 32682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-1/wal
uuid: "d5aa5495c77746a1b0c822b8c98f9081"
format_stamp: "Formatted at 2026-08-07 10:55:27 on dist-test-slave-x5fp"
server_key: "07f3e12d45749392ed9fcb2ec5e1d6c5"
server_key_iv: "194c8909bc4a39372a24713c335f099d"
server_key_version: "encryptionkey@0"
I20260807 10:55:27.045403 32682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:55:27.066244 32682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:55:27.070825 32682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:55:27.071200 32682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:55:27.072270 32682 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:55:27.073792 32682 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:55:27.073912 32682 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:27.074074 32682 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:55:27.074150 32682 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:27.110621 32682 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.194:37815
I20260807 10:55:27.110666 388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.194:37815 every 8 connection(s)
I20260807 10:55:27.112041 32682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
I20260807 10:55:27.118132 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 32682
I20260807 10:55:27.118366 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance
I20260807 10:55:27.118986 3367 external_mini_cluster.cc:1428] Setting key 2dd9cb076f5eb9b8c7b5e104efcbfcef
Aug 07 10:55:27 dist-test-slave-x5fp krb5kdc[32343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100127, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.194@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:55:27.137622 396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:27.114970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:52465 (local address 127.3.73.254:41493)
0807 10:55:27.115476 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:27.115482 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:27.116484 (+ 1002us) server_negotiation.cc:407] Connection header received
0807 10:55:27.118936 (+ 2452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:27.118941 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:27.119019 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:27.119170 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:27.120865 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:27.121918 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:27.123554 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:27.124351 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:27.124500 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:27.128933 (+ 4433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:27.128943 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:27.128951 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:55:27.129006 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:27.133408 (+ 4402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:27.134461 (+ 1053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:27.134470 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:27.134476 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:27.134646 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:27.135332 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:27.135339 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:27.135345 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:27.135780 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:27.135944 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:27.136807 (+ 863us) server_negotiation.cc:299] Negotiation successful
0807 10:55:27.137114 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":241,"threads_started":1}
I20260807 10:55:27.138536 397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:27.115515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41493 (local address 127.3.73.194:52465)
0807 10:55:27.116305 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:27.116349 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:27.118584 (+ 2235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:27.119461 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:27.119470 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:27.120024 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:55:27.120641 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:27.120662 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:27.122098 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:27.122102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:27.123392 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:27.123405 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:27.124578 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:27.124583 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:27.124879 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:27.125668 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:55:27.125703 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:55:27.128573 (+ 2870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:55:27.133648 (+ 5075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:55:27.133658 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:55:27.133676 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:55:27.134267 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:55:27.134847 (+ 580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:55:27.134852 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:55:27.134854 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:55:27.135097 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:55:27.135942 (+ 845us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:55:27.135949 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:55:27.136359 (+ 410us) client_negotiation.cc:769] Sending connection context
0807 10:55:27.136628 (+ 269us) client_negotiation.cc:240] Negotiation successful
0807 10:55:27.137194 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":294,"threads_started":1}
I20260807 10:55:27.140334 393 heartbeater.cc:347] Connected to a master server at 127.3.73.254:41493
I20260807 10:55:27.140877 393 heartbeater.cc:464] Registering TS with master...
I20260807 10:55:27.141937 393 heartbeater.cc:511] Master 127.3.73.254:41493 requested a full tablet report, sending...
I20260807 10:55:27.143893 32409 ts_manager.cc:194] Registered new tserver with Master: d5aa5495c77746a1b0c822b8c98f9081 (127.3.73.194:37815)
I20260807 10:55:27.145294 32409 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.194@KRBTEST.COM'} at 127.3.73.194:52465
WARNING: no policy specified for kudu/127.3.73.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab.
Entry for principal kudu/127.3.73.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab.
I20260807 10:55:27.204275 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.195:0
--local_ip_for_outbound_sockets=127.3.73.195
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:41493
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:39973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 10:55:27.545728 32675 heartbeater.cc:503] Master 127.3.73.254:41493 was elected leader, sending a full tablet report...
W20260807 10:55:27.603375 402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:55:27.603715 402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:55:27.603904 402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:55:27.611627 402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:55:27.611806 402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:55:27.611992 402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.195
I20260807 10:55:27.622568 402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:39973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.195.keytab
--principal=kudu/127.3.73.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:41493
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.195
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:55:27.624071 402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:55:27.625578 402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:55:27.640130 411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:27.640347 410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:27.640322 413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:27.641264 402 server_base.cc:1034] running on GCE node
I20260807 10:55:27.641969 402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:55:27.643159 402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:55:27.644441 402 hybrid_clock.cc:648] HybridClock initialized: now 1786100127644372 us; error 65 us; skew 500 ppm
I20260807 10:55:27.644886 402 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:55:27 dist-test-slave-x5fp krb5kdc[32343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100127, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:55:27.653733 402 init.cc:377] Logged in from keytab as kudu/127.3.73.195@KRBTEST.COM (short username kudu)
I20260807 10:55:27.655853 402 webserver.cc:460] Webserver started at http://127.3.73.195:35099/ using document root <none> and password file <none>
I20260807 10:55:27.656662 402 fs_manager.cc:359] Metadata directory not provided
I20260807 10:55:27.656881 402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:55:27.657275 402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:55:27.660563 402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-2/data/instance:
uuid: "20c403c08658480992ddb7f84da57967"
format_stamp: "Formatted at 2026-08-07 10:55:27 on dist-test-slave-x5fp"
server_key: "c7809f2b9f6821dfeb99993b7828110f"
server_key_iv: "09af066853a5b66ead1273d5f80c8bad"
server_key_version: "encryptionkey@0"
I20260807 10:55:27.661590 402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance:
uuid: "20c403c08658480992ddb7f84da57967"
format_stamp: "Formatted at 2026-08-07 10:55:27 on dist-test-slave-x5fp"
server_key: "c7809f2b9f6821dfeb99993b7828110f"
server_key_iv: "09af066853a5b66ead1273d5f80c8bad"
server_key_version: "encryptionkey@0"
I20260807 10:55:27.667778 402 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.002s
I20260807 10:55:27.671669 423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:27.673522 402 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 10:55:27.673729 402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-2/wal
uuid: "20c403c08658480992ddb7f84da57967"
format_stamp: "Formatted at 2026-08-07 10:55:27 on dist-test-slave-x5fp"
server_key: "c7809f2b9f6821dfeb99993b7828110f"
server_key_iv: "09af066853a5b66ead1273d5f80c8bad"
server_key_version: "encryptionkey@0"
I20260807 10:55:27.673985 402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:55:27.696915 402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:55:27.702018 402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:55:27.702353 402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:55:27.703467 402 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:55:27.705178 402 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:55:27.705296 402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:27.705456 402 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:55:27.705543 402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:27.742882 402 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.195:42761
I20260807 10:55:27.742898 592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.195:42761 every 8 connection(s)
I20260807 10:55:27.744297 402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
I20260807 10:55:27.745100 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 402
I20260807 10:55:27.745383 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance
I20260807 10:55:27.746074 3367 external_mini_cluster.cc:1428] Setting key edaab501b5420bf5c1b3b31152023b25
Aug 07 10:55:27 dist-test-slave-x5fp krb5kdc[32343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100127, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.195@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:55:27.767838 597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:27.747542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:39775 (local address 127.3.73.254:41493)
0807 10:55:27.748058 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:27.748065 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:27.749451 (+ 1386us) server_negotiation.cc:407] Connection header received
0807 10:55:27.751624 (+ 2173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:27.751629 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:27.751739 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:27.751870 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:27.753531 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:27.754326 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:27.755902 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:27.756317 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:27.756457 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:27.760458 (+ 4001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:27.760465 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:27.760470 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:55:27.760513 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:27.764027 (+ 3514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:27.765042 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:27.765046 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:27.765049 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:27.765184 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:27.765734 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:27.765738 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:27.765741 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:27.766151 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:27.766304 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:27.767254 (+ 950us) server_negotiation.cc:299] Negotiation successful
0807 10:55:27.767497 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":238,"threads_started":1}
I20260807 10:55:27.769029 598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:27.748147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:41493 (local address 127.3.73.195:39775)
0807 10:55:27.749270 (+ 1123us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:27.749329 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:27.751346 (+ 2017us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:27.752124 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:27.752133 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:27.752692 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:55:27.753355 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:27.753373 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:27.754497 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:27.754501 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:27.755753 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:27.755766 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:27.756574 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:27.756581 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:27.756894 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:27.757596 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:55:27.757624 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:55:27.760033 (+ 2409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:55:27.764208 (+ 4175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:55:27.764218 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:55:27.764235 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:55:27.764880 (+ 645us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:55:27.765320 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:55:27.765324 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:55:27.765327 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:55:27.765552 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:55:27.766321 (+ 769us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:55:27.766329 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:55:27.766774 (+ 445us) client_negotiation.cc:769] Sending connection context
0807 10:55:27.767074 (+ 300us) client_negotiation.cc:240] Negotiation successful
0807 10:55:27.767673 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"thread_start_us":372,"threads_started":1}
I20260807 10:55:27.770876 595 heartbeater.cc:347] Connected to a master server at 127.3.73.254:41493
I20260807 10:55:27.771270 595 heartbeater.cc:464] Registering TS with master...
I20260807 10:55:27.772260 595 heartbeater.cc:511] Master 127.3.73.254:41493 requested a full tablet report, sending...
I20260807 10:55:27.774453 32409 ts_manager.cc:194] Registered new tserver with Master: 20c403c08658480992ddb7f84da57967 (127.3.73.195:42761)
I20260807 10:55:27.775599 32409 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.73.195@KRBTEST.COM'} at 127.3.73.195:39775
I20260807 10:55:27.776916 3367 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 10:55:27.798102 597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:27.782622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46756 (local address 127.3.73.254:41493)
0807 10:55:27.782848 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:27.782855 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:27.783442 (+ 587us) server_negotiation.cc:407] Connection header received
0807 10:55:27.783611 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:27.783617 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:27.783730 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:27.783865 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:27.785624 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:27.786983 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:27.788365 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:27.789206 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:27.789336 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:27.792138 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:27.792143 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:27.792147 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:55:27.792187 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:27.795189 (+ 3002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:27.796001 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:27.796006 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:27.796008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:27.796150 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:27.796526 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:27.796530 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:27.796532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:27.797040 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:27.797239 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:27.797606 (+ 367us) server_negotiation.cc:299] Negotiation successful
0807 10:55:27.797856 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 10:55:27.802875 32408 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46756:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 10:55:27.810860 32408 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 10:55:27.855849 611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:27.826428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:37815 (local address 127.0.0.1:48674)
0807 10:55:27.844524 (+ 18096us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:27.844593 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:27.844886 (+ 293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:27.847352 (+ 2466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:27.847363 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:27.847394 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:55:27.848027 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:27.848043 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:27.850495 (+ 2452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:27.850500 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:27.852791 (+ 2291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:27.852808 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:27.854699 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:27.854705 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:27.854919 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:27.854974 (+ 55us) client_negotiation.cc:769] Sending connection context
0807 10:55:27.855132 (+ 158us) client_negotiation.cc:240] Negotiation successful
0807 10:55:27.855350 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17734,"mutex_wait_us":65,"thread_start_us":16619,"threads_started":1}
I20260807 10:55:27.856374 614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:27.826483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:33757 (local address 127.0.0.1:58244)
0807 10:55:27.846250 (+ 19767us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:27.846270 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:27.846524 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:27.847626 (+ 1102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:27.847630 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:27.847651 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:55:27.848626 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:27.848642 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:27.850643 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:27.850648 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:27.853089 (+ 2441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:27.853104 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:27.855395 (+ 2291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:27.855401 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:27.855603 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:27.855637 (+ 34us) client_negotiation.cc:769] Sending connection context
0807 10:55:27.855733 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 10:55:27.855898 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19630,"mutex_wait_us":133,"spinlock_wait_cycles":4480,"thread_start_us":19592,"threads_started":1}
I20260807 10:55:27.857283 615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:27.826721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:42761 (local address 127.0.0.1:36838)
0807 10:55:27.848524 (+ 21803us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:27.848554 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:27.848731 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:27.849407 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:27.849412 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:27.849438 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:55:27.849861 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:27.849874 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:27.851812 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:27.851818 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:27.854057 (+ 2239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:27.854074 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:27.856515 (+ 2441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:27.856522 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:27.856736 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:27.856771 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 10:55:27.856852 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 10:55:27.856973 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21659,"thread_start_us":21457,"threads_started":1}
I20260807 10:55:27.857290 618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:27.828005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58244 (local address 127.3.73.193:33757)
0807 10:55:27.839029 (+ 11024us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:27.839052 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:27.846401 (+ 7349us) server_negotiation.cc:407] Connection header received
0807 10:55:27.846762 (+ 361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:27.846769 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:27.847047 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:27.847246 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:55:27.848880 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:27.850445 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:27.853339 (+ 2894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:27.854874 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:27.854986 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:27.855209 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:27.856696 (+ 1487us) server_negotiation.cc:299] Negotiation successful
0807 10:55:27.856917 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10721,"thread_start_us":10502,"threads_started":1}
I20260807 10:55:27.858345 616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:27.827144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36838 (local address 127.3.73.195:42761)
0807 10:55:27.845779 (+ 18635us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:27.845791 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:27.848797 (+ 3006us) server_negotiation.cc:407] Connection header received
0807 10:55:27.848922 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:27.848929 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:27.849192 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:27.849415 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:55:27.850041 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:27.851639 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:27.854270 (+ 2631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:27.855768 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:27.855884 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:27.857781 (+ 1897us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:27.857854 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 10:55:27.858015 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18430,"thread_start_us":18266,"threads_started":1}
I20260807 10:55:27.858762 617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:27.827331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48674 (local address 127.3.73.194:37815)
0807 10:55:27.846143 (+ 18812us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:27.846151 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:27.846643 (+ 492us) server_negotiation.cc:407] Connection header received
0807 10:55:27.846755 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:27.846764 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:27.847047 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:27.847810 (+ 763us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:55:27.848390 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:27.850350 (+ 1960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:27.853014 (+ 2664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:27.854489 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:27.854658 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:27.854892 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:27.855180 (+ 288us) server_negotiation.cc:299] Negotiation successful
0807 10:55:27.855370 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18634,"thread_start_us":18490,"threads_started":1}
I20260807 10:55:27.862145 501 tablet_service.cc:1467] Processing CreateTablet for tablet 6e86eed705914f36a85e19cad6891bb7 (DEFAULT_TABLE table=test-table [id=6f6cf5f7bca24af99aa2eac8affc29c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:55:27.862195 301 tablet_service.cc:1467] Processing CreateTablet for tablet 6e86eed705914f36a85e19cad6891bb7 (DEFAULT_TABLE table=test-table [id=6f6cf5f7bca24af99aa2eac8affc29c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:55:27.862447 32580 tablet_service.cc:1467] Processing CreateTablet for tablet 6e86eed705914f36a85e19cad6891bb7 (DEFAULT_TABLE table=test-table [id=6f6cf5f7bca24af99aa2eac8affc29c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:55:27.863585 501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e86eed705914f36a85e19cad6891bb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:55:27.863706 301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e86eed705914f36a85e19cad6891bb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:55:27.863734 32580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e86eed705914f36a85e19cad6891bb7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:55:27.875530 619 tablet_bootstrap.cc:492] T 6e86eed705914f36a85e19cad6891bb7 P 20c403c08658480992ddb7f84da57967: Bootstrap starting.
I20260807 10:55:27.879496 622 tablet_bootstrap.cc:492] T 6e86eed705914f36a85e19cad6891bb7 P d5aa5495c77746a1b0c822b8c98f9081: Bootstrap starting.
I20260807 10:55:27.879774 619 tablet_bootstrap.cc:654] T 6e86eed705914f36a85e19cad6891bb7 P 20c403c08658480992ddb7f84da57967: Neither blocks nor log segments found. Creating new log.
I20260807 10:55:27.881682 620 tablet_bootstrap.cc:492] T 6e86eed705914f36a85e19cad6891bb7 P 191f08df78124986acbc92fc3ab9d2d3: Bootstrap starting.
I20260807 10:55:27.883677 622 tablet_bootstrap.cc:654] T 6e86eed705914f36a85e19cad6891bb7 P d5aa5495c77746a1b0c822b8c98f9081: Neither blocks nor log segments found. Creating new log.
I20260807 10:55:27.883643 619 log.cc:824] T 6e86eed705914f36a85e19cad6891bb7 P 20c403c08658480992ddb7f84da57967: Log is configured to *not* fsync() on all Append() calls
I20260807 10:55:27.885010 622 log.cc:824] T 6e86eed705914f36a85e19cad6891bb7 P d5aa5495c77746a1b0c822b8c98f9081: Log is configured to *not* fsync() on all Append() calls
I20260807 10:55:27.885417 620 tablet_bootstrap.cc:654] T 6e86eed705914f36a85e19cad6891bb7 P 191f08df78124986acbc92fc3ab9d2d3: Neither blocks nor log segments found. Creating new log.
I20260807 10:55:27.886435 620 log.cc:824] T 6e86eed705914f36a85e19cad6891bb7 P 191f08df78124986acbc92fc3ab9d2d3: Log is configured to *not* fsync() on all Append() calls
I20260807 10:55:27.888002 619 tablet_bootstrap.cc:492] T 6e86eed705914f36a85e19cad6891bb7 P 20c403c08658480992ddb7f84da57967: No bootstrap required, opened a new log
I20260807 10:55:27.888406 619 ts_tablet_manager.cc:1397] T 6e86eed705914f36a85e19cad6891bb7 P 20c403c08658480992ddb7f84da57967: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.000s
I20260807 10:55:27.889454 620 tablet_bootstrap.cc:492] T 6e86eed705914f36a85e19cad6891bb7 P 191f08df78124986acbc92fc3ab9d2d3: No bootstrap required, opened a new log
I20260807 10:55:27.889767 620 ts_tablet_manager.cc:1397] T 6e86eed705914f36a85e19cad6891bb7 P 191f08df78124986acbc92fc3ab9d2d3: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.001s
I20260807 10:55:27.896246 622 tablet_bootstrap.cc:492] T 6e86eed705914f36a85e19cad6891bb7 P d5aa5495c77746a1b0c822b8c98f9081: No bootstrap required, opened a new log
I20260807 10:55:27.896605 622 ts_tablet_manager.cc:1397] T 6e86eed705914f36a85e19cad6891bb7 P d5aa5495c77746a1b0c822b8c98f9081: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.004s
I20260807 10:55:27.898496 620 raft_consensus.cc:348] T 6e86eed705914f36a85e19cad6891bb7 P 191f08df78124986acbc92fc3ab9d2d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5aa5495c77746a1b0c822b8c98f9081" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 37815 } } peers { permanent_uuid: "191f08df78124986acbc92fc3ab9d2d3" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33757 } } peers { permanent_uuid: "20c403c08658480992ddb7f84da57967" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 42761 } }
I20260807 10:55:27.898981 620 raft_consensus.cc:374] T 6e86eed705914f36a85e19cad6891bb7 P 191f08df78124986acbc92fc3ab9d2d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:55:27.899086 620 raft_consensus.cc:729] T 6e86eed705914f36a85e19cad6891bb7 P 191f08df78124986acbc92fc3ab9d2d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 191f08df78124986acbc92fc3ab9d2d3, State: Initialized, Role: FOLLOWER
I20260807 10:55:27.899782 620 consensus_queue.cc:260] T 6e86eed705914f36a85e19cad6891bb7 P 191f08df78124986acbc92fc3ab9d2d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5aa5495c77746a1b0c822b8c98f9081" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 37815 } } peers { permanent_uuid: "191f08df78124986acbc92fc3ab9d2d3" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33757 } } peers { permanent_uuid: "20c403c08658480992ddb7f84da57967" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 42761 } }
I20260807 10:55:27.901237 619 raft_consensus.cc:348] T 6e86eed705914f36a85e19cad6891bb7 P 20c403c08658480992ddb7f84da57967 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5aa5495c77746a1b0c822b8c98f9081" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 37815 } } peers { permanent_uuid: "191f08df78124986acbc92fc3ab9d2d3" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33757 } } peers { permanent_uuid: "20c403c08658480992ddb7f84da57967" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 42761 } }
I20260807 10:55:27.901717 619 raft_consensus.cc:374] T 6e86eed705914f36a85e19cad6891bb7 P 20c403c08658480992ddb7f84da57967 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:55:27.901806 619 raft_consensus.cc:729] T 6e86eed705914f36a85e19cad6891bb7 P 20c403c08658480992ddb7f84da57967 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20c403c08658480992ddb7f84da57967, State: Initialized, Role: FOLLOWER
I20260807 10:55:27.901839 620 ts_tablet_manager.cc:1428] T 6e86eed705914f36a85e19cad6891bb7 P 191f08df78124986acbc92fc3ab9d2d3: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260807 10:55:27.902590 619 consensus_queue.cc:260] T 6e86eed705914f36a85e19cad6891bb7 P 20c403c08658480992ddb7f84da57967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5aa5495c77746a1b0c822b8c98f9081" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 37815 } } peers { permanent_uuid: "191f08df78124986acbc92fc3ab9d2d3" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33757 } } peers { permanent_uuid: "20c403c08658480992ddb7f84da57967" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 42761 } }
I20260807 10:55:27.904579 619 ts_tablet_manager.cc:1428] T 6e86eed705914f36a85e19cad6891bb7 P 20c403c08658480992ddb7f84da57967: Time spent starting tablet: real 0.016s user 0.001s sys 0.015s
I20260807 10:55:27.905433 595 heartbeater.cc:503] Master 127.3.73.254:41493 was elected leader, sending a full tablet report...
I20260807 10:55:27.907675 622 raft_consensus.cc:348] T 6e86eed705914f36a85e19cad6891bb7 P d5aa5495c77746a1b0c822b8c98f9081 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5aa5495c77746a1b0c822b8c98f9081" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 37815 } } peers { permanent_uuid: "191f08df78124986acbc92fc3ab9d2d3" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33757 } } peers { permanent_uuid: "20c403c08658480992ddb7f84da57967" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 42761 } }
I20260807 10:55:27.908118 622 raft_consensus.cc:374] T 6e86eed705914f36a85e19cad6891bb7 P d5aa5495c77746a1b0c822b8c98f9081 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:55:27.908244 622 raft_consensus.cc:729] T 6e86eed705914f36a85e19cad6891bb7 P d5aa5495c77746a1b0c822b8c98f9081 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5aa5495c77746a1b0c822b8c98f9081, State: Initialized, Role: FOLLOWER
I20260807 10:55:27.908988 622 consensus_queue.cc:260] T 6e86eed705914f36a85e19cad6891bb7 P d5aa5495c77746a1b0c822b8c98f9081 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5aa5495c77746a1b0c822b8c98f9081" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 37815 } } peers { permanent_uuid: "191f08df78124986acbc92fc3ab9d2d3" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33757 } } peers { permanent_uuid: "20c403c08658480992ddb7f84da57967" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 42761 } }
I20260807 10:55:27.910073 393 heartbeater.cc:503] Master 127.3.73.254:41493 was elected leader, sending a full tablet report...
I20260807 10:55:27.910784 622 ts_tablet_manager.cc:1428] T 6e86eed705914f36a85e19cad6891bb7 P d5aa5495c77746a1b0c822b8c98f9081: Time spent starting tablet: real 0.014s user 0.016s sys 0.000s
I20260807 10:55:27.980674 631 raft_consensus.cc:482] T 6e86eed705914f36a85e19cad6891bb7 P d5aa5495c77746a1b0c822b8c98f9081 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 10:55:27.981110 631 raft_consensus.cc:504] T 6e86eed705914f36a85e19cad6891bb7 P d5aa5495c77746a1b0c822b8c98f9081 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5aa5495c77746a1b0c822b8c98f9081" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 37815 } } peers { permanent_uuid: "191f08df78124986acbc92fc3ab9d2d3" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33757 } } peers { permanent_uuid: "20c403c08658480992ddb7f84da57967" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 42761 } }
I20260807 10:55:27.982941 631 leader_election.cc:288] T 6e86eed705914f36a85e19cad6891bb7 P d5aa5495c77746a1b0c822b8c98f9081 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 191f08df78124986acbc92fc3ab9d2d3 (127.3.73.193:33757), 20c403c08658480992ddb7f84da57967 (127.3.73.195:42761)
I20260807 10:55:27.990809 616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:27.983415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:60471 (local address 127.3.73.195:42761)
0807 10:55:27.983679 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:27.983689 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:27.984614 (+ 925us) server_negotiation.cc:407] Connection header received
0807 10:55:27.984803 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:27.984809 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:27.984902 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:27.984992 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:55:27.985680 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:27.986677 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:27.988960 (+ 2283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:27.989755 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:27.989960 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:27.990039 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:27.990390 (+ 351us) server_negotiation.cc:299] Negotiation successful
0807 10:55:27.990534 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 10:55:27.990839 618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:27.983424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:42823 (local address 127.3.73.193:33757)
0807 10:55:27.983761 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:27.983770 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:27.984784 (+ 1014us) server_negotiation.cc:407] Connection header received
0807 10:55:27.984989 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:27.984996 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:27.985076 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:27.985182 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:55:27.985818 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:27.986714 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:27.988932 (+ 2218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:27.989744 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:27.989925 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:27.990025 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:27.990446 (+ 421us) server_negotiation.cc:299] Negotiation successful
0807 10:55:27.990577 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158}
I20260807 10:55:27.990975 635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:27.983401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:33757 (local address 127.3.73.194:42823)
0807 10:55:27.984632 (+ 1231us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:27.984652 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:27.984819 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:27.985211 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:27.985215 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:27.985235 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:55:27.985652 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:27.985664 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:27.986901 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:27.986907 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:27.988703 (+ 1796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:27.988769 (+ 66us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:27.989994 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:27.990001 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:27.990195 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:27.990266 (+ 71us) client_negotiation.cc:769] Sending connection context
0807 10:55:27.990389 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 10:55:27.990530 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1088,"mutex_wait_us":58,"thread_start_us":747,"threads_started":1}
I20260807 10:55:27.991560 633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:27.983239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:42761 (local address 127.3.73.194:60471)
0807 10:55:27.984456 (+ 1217us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:27.984486 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:27.984636 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:27.985027 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:27.985031 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:27.985055 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:55:27.985502 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:27.985513 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:27.986823 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:27.986831 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:27.988761 (+ 1930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:27.988784 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:27.990031 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:27.990039 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:27.990192 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:27.990239 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 10:55:27.990314 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 10:55:27.990489 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1087,"mutex_wait_us":36,"thread_start_us":744,"threads_started":1}
I20260807 10:55:27.992184 32615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e86eed705914f36a85e19cad6891bb7" candidate_uuid: "d5aa5495c77746a1b0c822b8c98f9081" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "191f08df78124986acbc92fc3ab9d2d3" is_pre_election: true
I20260807 10:55:27.992635 529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e86eed705914f36a85e19cad6891bb7" candidate_uuid: "d5aa5495c77746a1b0c822b8c98f9081" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c403c08658480992ddb7f84da57967" is_pre_election: true
I20260807 10:55:27.992874 32615 raft_consensus.cc:2454] T 6e86eed705914f36a85e19cad6891bb7 P 191f08df78124986acbc92fc3ab9d2d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5aa5495c77746a1b0c822b8c98f9081 in term 0.
I20260807 10:55:27.993214 529 raft_consensus.cc:2454] T 6e86eed705914f36a85e19cad6891bb7 P 20c403c08658480992ddb7f84da57967 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5aa5495c77746a1b0c822b8c98f9081 in term 0.
I20260807 10:55:27.993683 32709 leader_election.cc:302] T 6e86eed705914f36a85e19cad6891bb7 P d5aa5495c77746a1b0c822b8c98f9081 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 191f08df78124986acbc92fc3ab9d2d3, d5aa5495c77746a1b0c822b8c98f9081; no voters:
I20260807 10:55:27.994154 631 raft_consensus.cc:2781] T 6e86eed705914f36a85e19cad6891bb7 P d5aa5495c77746a1b0c822b8c98f9081 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 10:55:27.994318 631 raft_consensus.cc:482] T 6e86eed705914f36a85e19cad6891bb7 P d5aa5495c77746a1b0c822b8c98f9081 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 10:55:27.994441 631 raft_consensus.cc:3037] T 6e86eed705914f36a85e19cad6891bb7 P d5aa5495c77746a1b0c822b8c98f9081 [term 0 FOLLOWER]: Advancing to term 1
W20260807 10:55:27.996315 596 tablet.cc:2369] T 6e86eed705914f36a85e19cad6891bb7 P 20c403c08658480992ddb7f84da57967: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:55:27.996876 631 raft_consensus.cc:504] T 6e86eed705914f36a85e19cad6891bb7 P d5aa5495c77746a1b0c822b8c98f9081 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5aa5495c77746a1b0c822b8c98f9081" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 37815 } } peers { permanent_uuid: "191f08df78124986acbc92fc3ab9d2d3" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33757 } } peers { permanent_uuid: "20c403c08658480992ddb7f84da57967" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 42761 } }
I20260807 10:55:27.997605 631 leader_election.cc:288] T 6e86eed705914f36a85e19cad6891bb7 P d5aa5495c77746a1b0c822b8c98f9081 [CANDIDATE]: Term 1 election: Requested vote from peers 191f08df78124986acbc92fc3ab9d2d3 (127.3.73.193:33757), 20c403c08658480992ddb7f84da57967 (127.3.73.195:42761)
I20260807 10:55:27.998119 32615 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e86eed705914f36a85e19cad6891bb7" candidate_uuid: "d5aa5495c77746a1b0c822b8c98f9081" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "191f08df78124986acbc92fc3ab9d2d3"
I20260807 10:55:27.998368 32615 raft_consensus.cc:3037] T 6e86eed705914f36a85e19cad6891bb7 P 191f08df78124986acbc92fc3ab9d2d3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:55:27.998370 529 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e86eed705914f36a85e19cad6891bb7" candidate_uuid: "d5aa5495c77746a1b0c822b8c98f9081" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c403c08658480992ddb7f84da57967"
I20260807 10:55:27.998590 529 raft_consensus.cc:3037] T 6e86eed705914f36a85e19cad6891bb7 P 20c403c08658480992ddb7f84da57967 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:55:28.001119 32615 raft_consensus.cc:2454] T 6e86eed705914f36a85e19cad6891bb7 P 191f08df78124986acbc92fc3ab9d2d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5aa5495c77746a1b0c822b8c98f9081 in term 1.
I20260807 10:55:28.001348 529 raft_consensus.cc:2454] T 6e86eed705914f36a85e19cad6891bb7 P 20c403c08658480992ddb7f84da57967 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5aa5495c77746a1b0c822b8c98f9081 in term 1.
I20260807 10:55:28.001753 32709 leader_election.cc:302] T 6e86eed705914f36a85e19cad6891bb7 P d5aa5495c77746a1b0c822b8c98f9081 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 191f08df78124986acbc92fc3ab9d2d3, d5aa5495c77746a1b0c822b8c98f9081; no voters:
I20260807 10:55:28.002130 631 raft_consensus.cc:2781] T 6e86eed705914f36a85e19cad6891bb7 P d5aa5495c77746a1b0c822b8c98f9081 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:55:28.002640 631 raft_consensus.cc:686] T 6e86eed705914f36a85e19cad6891bb7 P d5aa5495c77746a1b0c822b8c98f9081 [term 1 LEADER]: Becoming Leader. State: Replica: d5aa5495c77746a1b0c822b8c98f9081, State: Running, Role: LEADER
I20260807 10:55:28.003237 631 consensus_queue.cc:237] T 6e86eed705914f36a85e19cad6891bb7 P d5aa5495c77746a1b0c822b8c98f9081 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5aa5495c77746a1b0c822b8c98f9081" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 37815 } } peers { permanent_uuid: "191f08df78124986acbc92fc3ab9d2d3" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33757 } } peers { permanent_uuid: "20c403c08658480992ddb7f84da57967" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 42761 } }
I20260807 10:55:28.008708 32409 catalog_manager.cc:5495] T 6e86eed705914f36a85e19cad6891bb7 P d5aa5495c77746a1b0c822b8c98f9081 reported cstate change: term changed from 0 to 1, leader changed from <none> to d5aa5495c77746a1b0c822b8c98f9081 (127.3.73.194). New cstate: current_term: 1 leader_uuid: "d5aa5495c77746a1b0c822b8c98f9081" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5aa5495c77746a1b0c822b8c98f9081" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 37815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "191f08df78124986acbc92fc3ab9d2d3" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20c403c08658480992ddb7f84da57967" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 42761 } health_report { overall_health: UNKNOWN } } }
W20260807 10:55:28.012845 32677 tablet.cc:2369] T 6e86eed705914f36a85e19cad6891bb7 P 191f08df78124986acbc92fc3ab9d2d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:55:28.039644 617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:28.032971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48684 (local address 127.3.73.194:37815)
0807 10:55:28.033171 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:28.033176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:28.033453 (+ 277us) server_negotiation.cc:407] Connection header received
0807 10:55:28.033595 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:28.033599 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:28.033697 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:28.033850 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:55:28.034385 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:28.035290 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:28.036870 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:28.037506 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:28.037618 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:28.038240 (+ 622us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:55:28.038997 (+ 757us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:55:28.039139 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:28.039287 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 10:55:28.039387 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 10:55:28.049790 529 raft_consensus.cc:1261] T 6e86eed705914f36a85e19cad6891bb7 P 20c403c08658480992ddb7f84da57967 [term 1 FOLLOWER]: Refusing update from remote peer d5aa5495c77746a1b0c822b8c98f9081: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:55:28.049798 32615 raft_consensus.cc:1261] T 6e86eed705914f36a85e19cad6891bb7 P 191f08df78124986acbc92fc3ab9d2d3 [term 1 FOLLOWER]: Refusing update from remote peer d5aa5495c77746a1b0c822b8c98f9081: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:55:28.051179 638 consensus_queue.cc:1035] T 6e86eed705914f36a85e19cad6891bb7 P d5aa5495c77746a1b0c822b8c98f9081 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20c403c08658480992ddb7f84da57967" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 42761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:55:28.051750 631 consensus_queue.cc:1035] T 6e86eed705914f36a85e19cad6891bb7 P d5aa5495c77746a1b0c822b8c98f9081 [LEADER]: Connected to new peer: Peer: permanent_uuid: "191f08df78124986acbc92fc3ab9d2d3" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:55:28.066659 647 mvcc.cc:205] Tried to move back new op lower bound from 7315866124478767104 to 7315866124305592320. Current Snapshot: MvccSnapshot[applied={T|T < 7315866124478767104}]
I20260807 10:55:28.079701 32409 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46756:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:55:28.080055 32409 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46756:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:55:28.084501 32409 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7d4a94a408064cb3ba526d92dbd9d089: Sending DeleteTablet for 3 replicas of tablet 6e86eed705914f36a85e19cad6891bb7
I20260807 10:55:28.086334 32580 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e86eed705914f36a85e19cad6891bb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:55:28 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:58244
I20260807 10:55:28.086351 301 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e86eed705914f36a85e19cad6891bb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:55:28 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:48674
I20260807 10:55:28.086334 501 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e86eed705914f36a85e19cad6891bb7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:55:28 UTC) from {username='kudu', principal='kudu/127.3.73.254@KRBTEST.COM'} at 127.0.0.1:36838
I20260807 10:55:28.092445 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 32498
I20260807 10:55:28.102363 662 tablet_replica.cc:330] T 6e86eed705914f36a85e19cad6891bb7 P 20c403c08658480992ddb7f84da57967: stopping tablet replica
I20260807 10:55:28.102687 663 tablet_replica.cc:330] T 6e86eed705914f36a85e19cad6891bb7 P d5aa5495c77746a1b0c822b8c98f9081: stopping tablet replica
I20260807 10:55:28.103104 662 raft_consensus.cc:2229] T 6e86eed705914f36a85e19cad6891bb7 P 20c403c08658480992ddb7f84da57967 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:55:28.103381 663 raft_consensus.cc:2229] T 6e86eed705914f36a85e19cad6891bb7 P d5aa5495c77746a1b0c822b8c98f9081 [term 1 LEADER]: Raft consensus shutting down.
I20260807 10:55:28.103873 662 raft_consensus.cc:2258] T 6e86eed705914f36a85e19cad6891bb7 P 20c403c08658480992ddb7f84da57967 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:55:28.104265 663 raft_consensus.cc:2258] T 6e86eed705914f36a85e19cad6891bb7 P d5aa5495c77746a1b0c822b8c98f9081 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:55:28.105137 660 tablet_replica.cc:330] T 6e86eed705914f36a85e19cad6891bb7 P 191f08df78124986acbc92fc3ab9d2d3: stopping tablet replica
I20260807 10:55:28.105561 660 raft_consensus.cc:2229] T 6e86eed705914f36a85e19cad6891bb7 P 191f08df78124986acbc92fc3ab9d2d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:55:28.106125 660 raft_consensus.cc:2258] T 6e86eed705914f36a85e19cad6891bb7 P 191f08df78124986acbc92fc3ab9d2d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:55:28.108443 660 ts_tablet_manager.cc:1905] T 6e86eed705914f36a85e19cad6891bb7 P 191f08df78124986acbc92fc3ab9d2d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:55:28.108943 662 ts_tablet_manager.cc:1905] T 6e86eed705914f36a85e19cad6891bb7 P 20c403c08658480992ddb7f84da57967: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:55:28.115474 662 ts_tablet_manager.cc:1918] T 6e86eed705914f36a85e19cad6891bb7 P 20c403c08658480992ddb7f84da57967: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:55:28.115693 662 log.cc:1196] T 6e86eed705914f36a85e19cad6891bb7 P 20c403c08658480992ddb7f84da57967: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-2/wal/wals/6e86eed705914f36a85e19cad6891bb7
I20260807 10:55:28.116288 662 ts_tablet_manager.cc:1939] T 6e86eed705914f36a85e19cad6891bb7 P 20c403c08658480992ddb7f84da57967: Deleting consensus metadata
I20260807 10:55:28.117410 663 ts_tablet_manager.cc:1905] T 6e86eed705914f36a85e19cad6891bb7 P d5aa5495c77746a1b0c822b8c98f9081: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:55:28.118454 32389 catalog_manager.cc:4841] TS 20c403c08658480992ddb7f84da57967 (127.3.73.195:42761): tablet 6e86eed705914f36a85e19cad6891bb7 (table test-table [id=6f6cf5f7bca24af99aa2eac8affc29c6]) successfully deleted
Aug 07 10:55:28 dist-test-slave-x5fp krb5kdc[32343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.193@KRBTEST.COM
I20260807 10:55:28.126770 663 ts_tablet_manager.cc:1918] T 6e86eed705914f36a85e19cad6891bb7 P d5aa5495c77746a1b0c822b8c98f9081: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:55:28.126987 663 log.cc:1196] T 6e86eed705914f36a85e19cad6891bb7 P d5aa5495c77746a1b0c822b8c98f9081: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-1/wal/wals/6e86eed705914f36a85e19cad6891bb7
I20260807 10:55:28.127774 663 ts_tablet_manager.cc:1939] T 6e86eed705914f36a85e19cad6891bb7 P d5aa5495c77746a1b0c822b8c98f9081: Deleting consensus metadata
I20260807 10:55:28.130327 660 ts_tablet_manager.cc:1918] T 6e86eed705914f36a85e19cad6891bb7 P 191f08df78124986acbc92fc3ab9d2d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:55:28.130582 660 log.cc:1196] T 6e86eed705914f36a85e19cad6891bb7 P 191f08df78124986acbc92fc3ab9d2d3: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786099767749326-3367-0/minicluster-data/ts-0/wal/wals/6e86eed705914f36a85e19cad6891bb7
I20260807 10:55:28.130734 618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:28.094173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58260 (local address 127.3.73.193:33757)
0807 10:55:28.094451 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:28.094462 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:28.107256 (+ 12794us) server_negotiation.cc:407] Connection header received
0807 10:55:28.107511 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:28.107517 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:28.107628 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:28.107756 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:28.109944 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:28.111223 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:28.113669 (+ 2446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:28.114488 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:28.114610 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:28.121564 (+ 6954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:28.121587 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:28.121608 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:55:28.121671 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:28.126240 (+ 4569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:28.127485 (+ 1245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:28.127501 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:28.127504 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:28.127721 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:28.128317 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:28.128323 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:28.128326 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:28.128895 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:28.129236 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:28.129895 (+ 659us) server_negotiation.cc:299] Negotiation successful
0807 10:55:28.130178 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 10:55:28.131258 660 ts_tablet_manager.cc:1939] T 6e86eed705914f36a85e19cad6891bb7 P 191f08df78124986acbc92fc3ab9d2d3: Deleting consensus metadata
I20260807 10:55:28.133037 32668 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:55:28.133883 32392 catalog_manager.cc:4841] TS 191f08df78124986acbc92fc3ab9d2d3 (127.3.73.193:33757): tablet 6e86eed705914f36a85e19cad6891bb7 (table test-table [id=6f6cf5f7bca24af99aa2eac8affc29c6]) successfully deleted
I20260807 10:55:28.136039 32390 catalog_manager.cc:4841] TS d5aa5495c77746a1b0c822b8c98f9081 (127.3.73.194:37815): tablet 6e86eed705914f36a85e19cad6891bb7 (table test-table [id=6f6cf5f7bca24af99aa2eac8affc29c6]) successfully deleted
I20260807 10:55:28.237923 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 32498
I20260807 10:55:28.268285 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 32682
Aug 07 10:55:28 dist-test-slave-x5fp krb5kdc[32343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.194@KRBTEST.COM
I20260807 10:55:28.289940 617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:28.269259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48694 (local address 127.3.73.194:37815)
0807 10:55:28.269511 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:28.269516 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:28.269567 (+ 51us) server_negotiation.cc:407] Connection header received
0807 10:55:28.269940 (+ 373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:28.269953 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:28.270077 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:28.270177 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:28.272069 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:28.273162 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:28.275433 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:28.275963 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:28.276091 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:28.282325 (+ 6234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:28.282346 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:28.282365 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:55:28.282417 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:28.286192 (+ 3775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:28.287475 (+ 1283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:28.287484 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:28.287487 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:28.287670 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:28.288135 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:28.288140 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:28.288151 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:28.288569 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:28.289018 (+ 449us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:28.289250 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 10:55:28.289539 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260807 10:55:28.290737 384 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:55:28.415484 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 32682
I20260807 10:55:28.448175 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 402
Aug 07 10:55:28 dist-test-slave-x5fp krb5kdc[32343](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100125, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.195@KRBTEST.COM
I20260807 10:55:28.466565 616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:28.449089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36846 (local address 127.3.73.195:42761)
0807 10:55:28.449291 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:28.449298 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:28.449427 (+ 129us) server_negotiation.cc:407] Connection header received
0807 10:55:28.449657 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:28.449663 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:28.449746 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:28.449868 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:28.451335 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:28.452093 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:28.453568 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:28.454001 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:28.454151 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:28.458966 (+ 4815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:28.458988 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:28.459006 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:55:28.459063 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:28.462416 (+ 3353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:28.463838 (+ 1422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:28.463848 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:28.463851 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:28.464029 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:28.464526 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:28.464531 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:28.464533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:28.465092 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:28.465351 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:28.465864 (+ 513us) server_negotiation.cc:299] Negotiation successful
0807 10:55:28.466189 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 10:55:28.467429 586 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:55:28.563951 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 402
I20260807 10:55:28.592959 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 32363
I20260807 10:55:28.594213 32455 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:55:28.713474 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 32363
2026-08-07T10:55:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (5211 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T10:55:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:55:28Z Disabled control of system clock
I20260807 10:55:28.831532 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:39409
--webserver_interface=127.3.73.254
--webserver_port=0
--builtin_ntp_servers=127.3.73.212:41193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:39409
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 10:55:29.213680 677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:55:29.214035 677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:55:29.214126 677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:55:29.221877 677 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:55:29.222015 677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:55:29.222055 677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:55:29.222110 677 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:55:29.222157 677 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:55:29.232345 677 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:41193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:39409
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:39409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:55:29.233865 677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:55:29.235898 677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:55:29.249078 685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:29.249082 684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:29.249254 687 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:29.250222 677 server_base.cc:1034] running on GCE node
I20260807 10:55:29.251252 677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:55:29.252970 677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:55:29.254230 677 hybrid_clock.cc:648] HybridClock initialized: now 1786100129254200 us; error 42 us; skew 500 ppm
I20260807 10:55:29.254622 677 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 10:55:29.257386 677 webserver.cc:460] Webserver started at http://127.3.73.254:33291/ using document root <none> and password file <none>
I20260807 10:55:29.258183 677 fs_manager.cc:359] Metadata directory not provided
I20260807 10:55:29.258332 677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:55:29.258852 677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:55:29.262233 677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "63c715a1f0254c5d8e88ea1cbf12c2b5"
format_stamp: "Formatted at 2026-08-07 10:55:29 on dist-test-slave-x5fp"
server_key: "bbf1bcfde408583b7116aa242b8e8d4e"
server_key_iv: "0710179ec05286e699597fa5da1dae00"
server_key_version: "encryptionkey@0"
I20260807 10:55:29.263165 677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "63c715a1f0254c5d8e88ea1cbf12c2b5"
format_stamp: "Formatted at 2026-08-07 10:55:29 on dist-test-slave-x5fp"
server_key: "bbf1bcfde408583b7116aa242b8e8d4e"
server_key_iv: "0710179ec05286e699597fa5da1dae00"
server_key_version: "encryptionkey@0"
I20260807 10:55:29.269341 677 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 10:55:29.273586 696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:29.275480 677 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 10:55:29.275769 677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "63c715a1f0254c5d8e88ea1cbf12c2b5"
format_stamp: "Formatted at 2026-08-07 10:55:29 on dist-test-slave-x5fp"
server_key: "bbf1bcfde408583b7116aa242b8e8d4e"
server_key_iv: "0710179ec05286e699597fa5da1dae00"
server_key_version: "encryptionkey@0"
I20260807 10:55:29.276073 677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:55:29.298503 677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:55:29.299674 677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:55:29.300076 677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:55:29.324124 677 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:39409
I20260807 10:55:29.324164 764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:39409 every 8 connection(s)
I20260807 10:55:29.325747 677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:55:29.329891 765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:55:29.330293 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 677
I20260807 10:55:29.330488 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:55:29.330969 3367 external_mini_cluster.cc:1428] Setting key 91db96d7ce2272115b3c800e01a4a764
I20260807 10:55:29.346845 765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63c715a1f0254c5d8e88ea1cbf12c2b5: Bootstrap starting.
I20260807 10:55:29.350684 772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:29.333162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42068 (local address 127.3.73.254:39409)
0807 10:55:29.334442 (+ 1280us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:29.334458 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:29.334518 (+ 60us) server_negotiation.cc:407] Connection header received
0807 10:55:29.337411 (+ 2893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:29.337453 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:29.338275 (+ 822us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:29.338830 (+ 555us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:29.340243 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:29.342366 (+ 2123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:29.344471 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:29.345723 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:29.345948 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:29.347531 (+ 1583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:29.347543 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:29.347565 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:29.348315 (+ 750us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:29.348441 (+ 126us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:29.348471 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:29.348588 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:29.348917 (+ 329us) server_negotiation.cc:299] Negotiation successful
0807 10:55:29.349328 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":710,"thread_start_us":463,"threads_started":1}
I20260807 10:55:29.351447 765 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63c715a1f0254c5d8e88ea1cbf12c2b5: Neither blocks nor log segments found. Creating new log.
I20260807 10:55:29.352872 765 log.cc:824] T 00000000000000000000000000000000 P 63c715a1f0254c5d8e88ea1cbf12c2b5: Log is configured to *not* fsync() on all Append() calls
I20260807 10:55:29.356137 765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63c715a1f0254c5d8e88ea1cbf12c2b5: No bootstrap required, opened a new log
I20260807 10:55:29.364362 765 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63c715a1f0254c5d8e88ea1cbf12c2b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63c715a1f0254c5d8e88ea1cbf12c2b5" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 39409 } }
I20260807 10:55:29.364975 765 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63c715a1f0254c5d8e88ea1cbf12c2b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:55:29.365113 765 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63c715a1f0254c5d8e88ea1cbf12c2b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63c715a1f0254c5d8e88ea1cbf12c2b5, State: Initialized, Role: FOLLOWER
I20260807 10:55:29.365842 765 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63c715a1f0254c5d8e88ea1cbf12c2b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63c715a1f0254c5d8e88ea1cbf12c2b5" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 39409 } }
I20260807 10:55:29.366094 765 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63c715a1f0254c5d8e88ea1cbf12c2b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:55:29.366258 765 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63c715a1f0254c5d8e88ea1cbf12c2b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:55:29.366448 765 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63c715a1f0254c5d8e88ea1cbf12c2b5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:55:29.368978 765 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63c715a1f0254c5d8e88ea1cbf12c2b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63c715a1f0254c5d8e88ea1cbf12c2b5" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 39409 } }
I20260807 10:55:29.369580 765 leader_election.cc:302] T 00000000000000000000000000000000 P 63c715a1f0254c5d8e88ea1cbf12c2b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63c715a1f0254c5d8e88ea1cbf12c2b5; no voters:
I20260807 10:55:29.370442 765 leader_election.cc:288] T 00000000000000000000000000000000 P 63c715a1f0254c5d8e88ea1cbf12c2b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:55:29.370702 775 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63c715a1f0254c5d8e88ea1cbf12c2b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:55:29.373945 775 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63c715a1f0254c5d8e88ea1cbf12c2b5 [term 1 LEADER]: Becoming Leader. State: Replica: 63c715a1f0254c5d8e88ea1cbf12c2b5, State: Running, Role: LEADER
I20260807 10:55:29.374603 775 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63c715a1f0254c5d8e88ea1cbf12c2b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63c715a1f0254c5d8e88ea1cbf12c2b5" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 39409 } }
I20260807 10:55:29.374935 765 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63c715a1f0254c5d8e88ea1cbf12c2b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:55:29.377977 776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63c715a1f0254c5d8e88ea1cbf12c2b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63c715a1f0254c5d8e88ea1cbf12c2b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63c715a1f0254c5d8e88ea1cbf12c2b5" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 39409 } } }
I20260807 10:55:29.378099 777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63c715a1f0254c5d8e88ea1cbf12c2b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63c715a1f0254c5d8e88ea1cbf12c2b5. Latest consensus state: current_term: 1 leader_uuid: "63c715a1f0254c5d8e88ea1cbf12c2b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63c715a1f0254c5d8e88ea1cbf12c2b5" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 39409 } } }
I20260807 10:55:29.378445 776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63c715a1f0254c5d8e88ea1cbf12c2b5 [sys.catalog]: This master's current role is: LEADER
I20260807 10:55:29.378566 777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63c715a1f0254c5d8e88ea1cbf12c2b5 [sys.catalog]: This master's current role is: LEADER
I20260807 10:55:29.380844 786 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:55:29.387697 786 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:55:29.398658 786 catalog_manager.cc:1347] Generated new cluster ID: 1a6b0919ee80406692754368c94a9aaa
I20260807 10:55:29.398816 786 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:55:29.422084 786 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:55:29.424165 786 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:55:29.435534 786 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63c715a1f0254c5d8e88ea1cbf12c2b5: Generated new TSK 0
I20260807 10:55:29.436834 786 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 10:55:29.445708 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.193:0
--local_ip_for_outbound_sockets=127.3.73.193
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:39409
--builtin_ntp_servers=127.3.73.212:41193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 10:55:29.846374 804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:55:29.846678 804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:55:29.846766 804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:55:29.854664 804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:55:29.854782 804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:55:29.854897 804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.193
I20260807 10:55:29.864533 804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:41193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.73.193
--webserver_port=0
--tserver_master_addrs=127.3.73.254:39409
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.193
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:55:29.866309 804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:55:29.868567 804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:55:29.883684 814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:29.883807 817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:29.883824 815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:29.885489 804 server_base.cc:1034] running on GCE node
I20260807 10:55:29.886219 804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:55:29.887346 804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:55:29.888623 804 hybrid_clock.cc:648] HybridClock initialized: now 1786100129888521 us; error 96 us; skew 500 ppm
I20260807 10:55:29.889096 804 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 10:55:29.891782 804 webserver.cc:460] Webserver started at http://127.3.73.193:42567/ using document root <none> and password file <none>
I20260807 10:55:29.892607 804 fs_manager.cc:359] Metadata directory not provided
I20260807 10:55:29.892760 804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:55:29.893150 804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:55:29.896378 804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-0/data/instance:
uuid: "a08e63525d52423bbbf6911a1573ccde"
format_stamp: "Formatted at 2026-08-07 10:55:29 on dist-test-slave-x5fp"
server_key: "bb59f6a9f64d90ad6b08d70a755a94ca"
server_key_iv: "ac23a6fe08cd29250d22e62eec0bf265"
server_key_version: "encryptionkey@0"
I20260807 10:55:29.897354 804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance:
uuid: "a08e63525d52423bbbf6911a1573ccde"
format_stamp: "Formatted at 2026-08-07 10:55:29 on dist-test-slave-x5fp"
server_key: "bb59f6a9f64d90ad6b08d70a755a94ca"
server_key_iv: "ac23a6fe08cd29250d22e62eec0bf265"
server_key_version: "encryptionkey@0"
I20260807 10:55:29.903388 804 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260807 10:55:29.908131 826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:29.910141 804 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 10:55:29.910429 804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-0/wal
uuid: "a08e63525d52423bbbf6911a1573ccde"
format_stamp: "Formatted at 2026-08-07 10:55:29 on dist-test-slave-x5fp"
server_key: "bb59f6a9f64d90ad6b08d70a755a94ca"
server_key_iv: "ac23a6fe08cd29250d22e62eec0bf265"
server_key_version: "encryptionkey@0"
I20260807 10:55:29.910740 804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:55:29.927335 804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:55:29.928323 804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:55:29.928676 804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:55:29.929881 804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:55:29.931619 804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:55:29.931766 804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:29.931908 804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:55:29.932004 804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:29.973420 804 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.193:46793
I20260807 10:55:29.973479 986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.193:46793 every 8 connection(s)
I20260807 10:55:29.974893 804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-0/data/info.pb
I20260807 10:55:29.976677 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 804
I20260807 10:55:29.976902 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-0/wal/instance
I20260807 10:55:29.977476 3367 external_mini_cluster.cc:1428] Setting key 9173dc83dc67ba874122fd205f70bee0
I20260807 10:55:29.980690 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.194:0
--local_ip_for_outbound_sockets=127.3.73.194
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:39409
--builtin_ntp_servers=127.3.73.212:41193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 10:55:29.990324 990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:29.978050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:49637 (local address 127.3.73.254:39409)
0807 10:55:29.978575 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:29.978580 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:29.980180 (+ 1600us) server_negotiation.cc:407] Connection header received
0807 10:55:29.982508 (+ 2328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:29.982517 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:29.982619 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:29.982750 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:29.983888 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:29.985102 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:29.986748 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:29.987463 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:29.987613 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:29.989205 (+ 1592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:29.989214 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:29.989221 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:29.989250 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:29.989304 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:29.989324 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:29.989435 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:29.989723 (+ 288us) server_negotiation.cc:299] Negotiation successful
0807 10:55:29.989851 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":283,"threads_started":1}
I20260807 10:55:29.991222 992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:29.978785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39409 (local address 127.3.73.193:49637)
0807 10:55:29.980006 (+ 1221us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:29.980056 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:29.982219 (+ 2163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:29.983006 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:29.983015 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:29.983106 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:55:29.983705 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:29.983722 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:29.985290 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:29.985294 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:29.986590 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:29.986603 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:29.987725 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:29.987731 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:29.987946 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:29.988849 (+ 903us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 10:55:29.988852 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:55:29.988896 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 10:55:29.988985 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:55:29.989484 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:55:29.989495 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:55:29.989503 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 10:55:29.989681 (+ 178us) client_negotiation.cc:240] Negotiation successful
0807 10:55:29.990022 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":404,"threads_started":1}
I20260807 10:55:29.993013 987 heartbeater.cc:347] Connected to a master server at 127.3.73.254:39409
I20260807 10:55:29.993394 987 heartbeater.cc:464] Registering TS with master...
I20260807 10:55:29.994424 987 heartbeater.cc:511] Master 127.3.73.254:39409 requested a full tablet report, sending...
I20260807 10:55:29.997097 717 ts_manager.cc:194] Registered new tserver with Master: a08e63525d52423bbbf6911a1573ccde (127.3.73.193:46793)
I20260807 10:55:29.998885 717 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.73.193:49637
W20260807 10:55:30.366647 993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:55:30.367031 993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:55:30.367133 993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:55:30.375027 993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:55:30.375242 993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:55:30.375370 993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.194
I20260807 10:55:30.386946 993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:41193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.73.194
--webserver_port=0
--tserver_master_addrs=127.3.73.254:39409
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.194
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:55:30.388417 993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:55:30.391050 993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:55:30.407404 1005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:30.407416 999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:30.407675 1000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:30.409339 993 server_base.cc:1034] running on GCE node
I20260807 10:55:30.410136 993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:55:30.411377 993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:55:30.412693 993 hybrid_clock.cc:648] HybridClock initialized: now 1786100130412611 us; error 70 us; skew 500 ppm
I20260807 10:55:30.413247 993 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 10:55:30.416821 993 webserver.cc:460] Webserver started at http://127.3.73.194:36007/ using document root <none> and password file <none>
I20260807 10:55:30.417691 993 fs_manager.cc:359] Metadata directory not provided
I20260807 10:55:30.417811 993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:55:30.418130 993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:55:30.421814 993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-1/data/instance:
uuid: "20ba5901030643c8ae30e251a0b3ea5c"
format_stamp: "Formatted at 2026-08-07 10:55:30 on dist-test-slave-x5fp"
server_key: "0f9c46a0afba1569c4edbefe8a33b53e"
server_key_iv: "2a3ff74cc1155d10f450726f33106257"
server_key_version: "encryptionkey@0"
I20260807 10:55:30.422832 993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance:
uuid: "20ba5901030643c8ae30e251a0b3ea5c"
format_stamp: "Formatted at 2026-08-07 10:55:30 on dist-test-slave-x5fp"
server_key: "0f9c46a0afba1569c4edbefe8a33b53e"
server_key_iv: "2a3ff74cc1155d10f450726f33106257"
server_key_version: "encryptionkey@0"
I20260807 10:55:30.429358 993 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 10:55:30.433970 1012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:30.436152 993 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 10:55:30.436419 993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-1/wal
uuid: "20ba5901030643c8ae30e251a0b3ea5c"
format_stamp: "Formatted at 2026-08-07 10:55:30 on dist-test-slave-x5fp"
server_key: "0f9c46a0afba1569c4edbefe8a33b53e"
server_key_iv: "2a3ff74cc1155d10f450726f33106257"
server_key_version: "encryptionkey@0"
I20260807 10:55:30.436726 993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:55:30.496084 993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:55:30.497243 993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:55:30.497615 993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:55:30.498831 993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:55:30.500633 993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:55:30.500808 993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:30.500967 993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:55:30.501078 993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:30.541263 993 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.194:45735
I20260807 10:55:30.541311 1178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.194:45735 every 8 connection(s)
I20260807 10:55:30.542675 993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-1/data/info.pb
I20260807 10:55:30.545356 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 993
I20260807 10:55:30.545532 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-1/wal/instance
I20260807 10:55:30.546068 3367 external_mini_cluster.cc:1428] Setting key 25b66c8a85903f43eec794d4a0199f14
I20260807 10:55:30.549185 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.73.195:0
--local_ip_for_outbound_sockets=127.3.73.195
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:39409
--builtin_ntp_servers=127.3.73.212:41193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 10:55:30.558461 1183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:30.545617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.194:58329 (local address 127.3.73.254:39409)
0807 10:55:30.546177 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:30.546186 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:30.547650 (+ 1464us) server_negotiation.cc:407] Connection header received
0807 10:55:30.549768 (+ 2118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:30.549776 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:30.549878 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:30.550006 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:30.551499 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:30.552703 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:30.554629 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:30.555383 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:30.555539 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:30.557355 (+ 1816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:30.557366 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:30.557373 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:30.557402 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:30.557463 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:30.557484 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:30.557591 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:30.557849 (+ 258us) server_negotiation.cc:299] Negotiation successful
0807 10:55:30.557974 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":268,"threads_started":1}
I20260807 10:55:30.559450 1184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:30.546252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39409 (local address 127.3.73.194:58329)
0807 10:55:30.547445 (+ 1193us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:30.547497 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:30.549466 (+ 1969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:30.550485 (+ 1019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:30.550495 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:30.550601 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:55:30.551292 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:30.551312 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:30.552942 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:30.552947 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:30.554442 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:30.554458 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:30.555680 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:30.555687 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:30.555974 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:30.556957 (+ 983us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 10:55:30.556961 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:55:30.557017 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 10:55:30.557121 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:55:30.557599 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:55:30.557609 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:55:30.557616 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 10:55:30.557827 (+ 211us) client_negotiation.cc:240] Negotiation successful
0807 10:55:30.558190 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":391,"threads_started":1}
I20260807 10:55:30.561338 1181 heartbeater.cc:347] Connected to a master server at 127.3.73.254:39409
I20260807 10:55:30.561730 1181 heartbeater.cc:464] Registering TS with master...
I20260807 10:55:30.562779 1181 heartbeater.cc:511] Master 127.3.73.254:39409 requested a full tablet report, sending...
I20260807 10:55:30.564841 717 ts_manager.cc:194] Registered new tserver with Master: 20ba5901030643c8ae30e251a0b3ea5c (127.3.73.194:45735)
I20260807 10:55:30.566087 717 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.73.194:58329
W20260807 10:55:30.939911 1185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:55:30.940258 1185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:55:30.940413 1185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:55:30.949023 1185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:55:30.949178 1185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:55:30.949405 1185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.73.195
I20260807 10:55:30.960311 1185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:41193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.73.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.73.195
--webserver_port=0
--tserver_master_addrs=127.3.73.254:39409
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.73.195
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:55:30.961952 1185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:55:30.964093 1185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:55:30.981739 1194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:30.981946 1196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:30.981972 1193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:30.982964 1185 server_base.cc:1034] running on GCE node
I20260807 10:55:30.983619 1185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:55:30.984673 1185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:55:30.985985 1185 hybrid_clock.cc:648] HybridClock initialized: now 1786100130985918 us; error 60 us; skew 500 ppm
I20260807 10:55:30.986392 1185 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 10:55:30.989161 1185 webserver.cc:460] Webserver started at http://127.3.73.195:43347/ using document root <none> and password file <none>
I20260807 10:55:30.989984 1185 fs_manager.cc:359] Metadata directory not provided
I20260807 10:55:30.990135 1185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:55:30.990520 1185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:55:30.994004 1185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-2/data/instance:
uuid: "1b541bf169b8418ca965c633c309b4a9"
format_stamp: "Formatted at 2026-08-07 10:55:30 on dist-test-slave-x5fp"
server_key: "bebb333a8722467dab6a2c267c9fb936"
server_key_iv: "53f1d73fb6b94f3725f1a19a0886a0c7"
server_key_version: "encryptionkey@0"
I20260807 10:55:30.994941 1185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance:
uuid: "1b541bf169b8418ca965c633c309b4a9"
format_stamp: "Formatted at 2026-08-07 10:55:30 on dist-test-slave-x5fp"
server_key: "bebb333a8722467dab6a2c267c9fb936"
server_key_iv: "53f1d73fb6b94f3725f1a19a0886a0c7"
server_key_version: "encryptionkey@0"
I20260807 10:55:31.000877 1185 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 10:55:31.002723 987 heartbeater.cc:503] Master 127.3.73.254:39409 was elected leader, sending a full tablet report...
I20260807 10:55:31.005566 1208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:31.007287 1185 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 10:55:31.007462 1185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-2/wal
uuid: "1b541bf169b8418ca965c633c309b4a9"
format_stamp: "Formatted at 2026-08-07 10:55:30 on dist-test-slave-x5fp"
server_key: "bebb333a8722467dab6a2c267c9fb936"
server_key_iv: "53f1d73fb6b94f3725f1a19a0886a0c7"
server_key_version: "encryptionkey@0"
I20260807 10:55:31.007678 1185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:55:31.021095 1185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:55:31.022028 1185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:55:31.022323 1185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:55:31.023435 1185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 10:55:31.025136 1185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 10:55:31.025239 1185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:31.025349 1185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 10:55:31.025409 1185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:31.062364 1185 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.195:37639
I20260807 10:55:31.062455 1373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.195:37639 every 8 connection(s)
I20260807 10:55:31.063758 1185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-2/data/info.pb
I20260807 10:55:31.073520 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 1185
I20260807 10:55:31.073812 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-2/wal/instance
I20260807 10:55:31.074381 3367 external_mini_cluster.cc:1428] Setting key 94911910ad086c578140060c56b5931c
I20260807 10:55:31.078572 1376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:31.066489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.195:50431 (local address 127.3.73.254:39409)
0807 10:55:31.066958 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:31.066962 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:31.068114 (+ 1152us) server_negotiation.cc:407] Connection header received
0807 10:55:31.070085 (+ 1971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:31.070091 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:31.070186 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:31.070308 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:31.071495 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:31.072270 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:31.074361 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:31.074903 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:31.075062 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:31.077440 (+ 2378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:31.077447 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:31.077452 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:31.077476 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:31.077534 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:31.077554 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:31.077667 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:31.078065 (+ 398us) server_negotiation.cc:299] Negotiation successful
0807 10:55:31.078212 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":251,"threads_started":1}
I20260807 10:55:31.079674 1377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:31.066968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:39409 (local address 127.3.73.195:50431)
0807 10:55:31.067941 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:31.067981 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:31.069778 (+ 1797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:31.070575 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:31.070584 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:31.070681 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:55:31.071305 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:31.071325 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:31.072471 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:31.072475 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:31.074145 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:31.074163 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:31.075234 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:31.075245 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:31.075715 (+ 470us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:31.076958 (+ 1243us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 10:55:31.076962 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:55:31.077063 (+ 101us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 10:55:31.077169 (+ 106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:55:31.077705 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:55:31.077716 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:55:31.077723 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 10:55:31.078007 (+ 284us) client_negotiation.cc:240] Negotiation successful
0807 10:55:31.078477 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":312,"threads_started":1}
I20260807 10:55:31.082259 1374 heartbeater.cc:347] Connected to a master server at 127.3.73.254:39409
I20260807 10:55:31.082669 1374 heartbeater.cc:464] Registering TS with master...
I20260807 10:55:31.083873 1374 heartbeater.cc:511] Master 127.3.73.254:39409 requested a full tablet report, sending...
I20260807 10:55:31.085884 717 ts_manager.cc:194] Registered new tserver with Master: 1b541bf169b8418ca965c633c309b4a9 (127.3.73.195:37639)
I20260807 10:55:31.086885 717 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.73.195:50431
I20260807 10:55:31.091755 3367 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 10:55:31.103600 1376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:31.096851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59058 (local address 127.3.73.254:39409)
0807 10:55:31.097051 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:31.097055 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:31.097353 (+ 298us) server_negotiation.cc:407] Connection header received
0807 10:55:31.097572 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:31.097577 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:31.097654 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:31.097762 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:31.098520 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:31.099256 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:31.100815 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:31.101278 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:31.101401 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:31.102858 (+ 1457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:31.102863 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:31.102866 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:31.102889 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:31.102926 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:31.102941 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:31.103029 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:31.103192 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 10:55:31.103301 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 10:55:31.113986 717 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59058:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 10:55:31.121942 717 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 10:55:31.150986 1396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:31.136874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35558 (local address 127.3.73.193:46793)
0807 10:55:31.140623 (+ 3749us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:31.140632 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:31.140663 (+ 31us) server_negotiation.cc:407] Connection header received
0807 10:55:31.140804 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:31.140814 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:31.141127 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:31.141413 (+ 286us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:55:31.142513 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:31.144260 (+ 1747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:31.147790 (+ 3530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:31.149669 (+ 1879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:31.149852 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:31.150181 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:31.150399 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 10:55:31.150589 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3556,"thread_start_us":3404,"threads_started":1}
I20260807 10:55:31.151014 1393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:31.136497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.193:46793 (local address 127.0.0.1:35558)
0807 10:55:31.137987 (+ 1490us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:31.138161 (+ 174us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:31.138441 (+ 280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:31.141528 (+ 3087us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:31.141538 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:31.141578 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:55:31.142145 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:31.142162 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:31.144411 (+ 2249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:31.144418 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:31.147484 (+ 3066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:31.147508 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:31.149903 (+ 2395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:31.149909 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:31.150111 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:31.150179 (+ 68us) client_negotiation.cc:769] Sending connection context
0807 10:55:31.150337 (+ 158us) client_negotiation.cc:240] Negotiation successful
0807 10:55:31.150562 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":909,"mutex_wait_us":1,"thread_start_us":1113,"threads_started":1}
I20260807 10:55:31.152858 1397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:31.136976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:37639 (local address 127.0.0.1:55552)
0807 10:55:31.140457 (+ 3481us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:31.140480 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:31.140645 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:31.143519 (+ 2874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:31.143524 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:31.143552 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:55:31.143946 (+ 394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:31.143959 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:31.145745 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:31.145754 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:31.148474 (+ 2720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:31.148488 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:31.150700 (+ 2212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:31.150708 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:31.150895 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:31.150944 (+ 49us) client_negotiation.cc:769] Sending connection context
0807 10:55:31.152217 (+ 1273us) client_negotiation.cc:240] Negotiation successful
0807 10:55:31.152396 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3342,"mutex_wait_us":4,"thread_start_us":3159,"threads_started":1}
I20260807 10:55:31.153051 1404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:31.140930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46904 (local address 127.3.73.194:45735)
0807 10:55:31.142881 (+ 1951us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:31.142903 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:31.142940 (+ 37us) server_negotiation.cc:407] Connection header received
0807 10:55:31.143044 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:31.143052 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:31.143327 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:31.143559 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:55:31.144692 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:31.146488 (+ 1796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:31.149544 (+ 3056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:31.151208 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:31.151367 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:31.151667 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:31.152352 (+ 685us) server_negotiation.cc:299] Negotiation successful
0807 10:55:31.152592 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1781,"thread_start_us":1665,"threads_started":1}
I20260807 10:55:31.153733 1400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:31.140875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:45735 (local address 127.0.0.1:46904)
0807 10:55:31.141529 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:31.141547 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:31.141665 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:31.143826 (+ 2161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:31.143852 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:31.143937 (+ 85us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:55:31.144431 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:31.144447 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:31.146681 (+ 2234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:31.146688 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:31.149295 (+ 2607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:31.149311 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:31.151816 (+ 2505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:31.151823 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:31.152029 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:31.152064 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 10:55:31.152164 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 10:55:31.153355 (+ 1191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"mutex_wait_us":992,"thread_start_us":301,"threads_started":1}
I20260807 10:55:31.153877 1398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:31.137166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55552 (local address 127.3.73.195:37639)
0807 10:55:31.140106 (+ 2940us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:31.140118 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:31.140629 (+ 511us) server_negotiation.cc:407] Connection header received
0807 10:55:31.142574 (+ 1945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:31.142585 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:31.142958 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:31.143213 (+ 255us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:55:31.144167 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:31.145584 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:31.148702 (+ 3118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:31.150069 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:31.150227 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:31.150540 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:31.153209 (+ 2669us) server_negotiation.cc:299] Negotiation successful
0807 10:55:31.153429 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2741,"thread_start_us":2559,"threads_started":1}
I20260807 10:55:31.155968 896 tablet_service.cc:1467] Processing CreateTablet for tablet f273f8f25c514ec58f7250016dede3f9 (DEFAULT_TABLE table=test-table [id=7b7c92686c864e13aeed03501b1289fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:55:31.157861 896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f273f8f25c514ec58f7250016dede3f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:55:31.157809 1083 tablet_service.cc:1467] Processing CreateTablet for tablet f273f8f25c514ec58f7250016dede3f9 (DEFAULT_TABLE table=test-table [id=7b7c92686c864e13aeed03501b1289fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:55:31.157819 1281 tablet_service.cc:1467] Processing CreateTablet for tablet f273f8f25c514ec58f7250016dede3f9 (DEFAULT_TABLE table=test-table [id=7b7c92686c864e13aeed03501b1289fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 10:55:31.159251 1281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f273f8f25c514ec58f7250016dede3f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:55:31.159322 1083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f273f8f25c514ec58f7250016dede3f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:55:31.174135 1405 tablet_bootstrap.cc:492] T f273f8f25c514ec58f7250016dede3f9 P 1b541bf169b8418ca965c633c309b4a9: Bootstrap starting.
I20260807 10:55:31.177593 1406 tablet_bootstrap.cc:492] T f273f8f25c514ec58f7250016dede3f9 P 20ba5901030643c8ae30e251a0b3ea5c: Bootstrap starting.
I20260807 10:55:31.178706 1407 tablet_bootstrap.cc:492] T f273f8f25c514ec58f7250016dede3f9 P a08e63525d52423bbbf6911a1573ccde: Bootstrap starting.
I20260807 10:55:31.179224 1405 tablet_bootstrap.cc:654] T f273f8f25c514ec58f7250016dede3f9 P 1b541bf169b8418ca965c633c309b4a9: Neither blocks nor log segments found. Creating new log.
I20260807 10:55:31.180783 1405 log.cc:824] T f273f8f25c514ec58f7250016dede3f9 P 1b541bf169b8418ca965c633c309b4a9: Log is configured to *not* fsync() on all Append() calls
I20260807 10:55:31.183418 1407 tablet_bootstrap.cc:654] T f273f8f25c514ec58f7250016dede3f9 P a08e63525d52423bbbf6911a1573ccde: Neither blocks nor log segments found. Creating new log.
I20260807 10:55:31.183418 1406 tablet_bootstrap.cc:654] T f273f8f25c514ec58f7250016dede3f9 P 20ba5901030643c8ae30e251a0b3ea5c: Neither blocks nor log segments found. Creating new log.
I20260807 10:55:31.185186 1407 log.cc:824] T f273f8f25c514ec58f7250016dede3f9 P a08e63525d52423bbbf6911a1573ccde: Log is configured to *not* fsync() on all Append() calls
I20260807 10:55:31.185186 1406 log.cc:824] T f273f8f25c514ec58f7250016dede3f9 P 20ba5901030643c8ae30e251a0b3ea5c: Log is configured to *not* fsync() on all Append() calls
I20260807 10:55:31.188301 1405 tablet_bootstrap.cc:492] T f273f8f25c514ec58f7250016dede3f9 P 1b541bf169b8418ca965c633c309b4a9: No bootstrap required, opened a new log
I20260807 10:55:31.188547 1406 tablet_bootstrap.cc:492] T f273f8f25c514ec58f7250016dede3f9 P 20ba5901030643c8ae30e251a0b3ea5c: No bootstrap required, opened a new log
I20260807 10:55:31.188776 1405 ts_tablet_manager.cc:1397] T f273f8f25c514ec58f7250016dede3f9 P 1b541bf169b8418ca965c633c309b4a9: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.010s
I20260807 10:55:31.188968 1406 ts_tablet_manager.cc:1397] T f273f8f25c514ec58f7250016dede3f9 P 20ba5901030643c8ae30e251a0b3ea5c: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.005s
I20260807 10:55:31.189117 1407 tablet_bootstrap.cc:492] T f273f8f25c514ec58f7250016dede3f9 P a08e63525d52423bbbf6911a1573ccde: No bootstrap required, opened a new log
I20260807 10:55:31.189421 1407 ts_tablet_manager.cc:1397] T f273f8f25c514ec58f7250016dede3f9 P a08e63525d52423bbbf6911a1573ccde: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.008s
I20260807 10:55:31.197434 1406 raft_consensus.cc:348] T f273f8f25c514ec58f7250016dede3f9 P 20ba5901030643c8ae30e251a0b3ea5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a08e63525d52423bbbf6911a1573ccde" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 46793 } } peers { permanent_uuid: "1b541bf169b8418ca965c633c309b4a9" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 37639 } } peers { permanent_uuid: "20ba5901030643c8ae30e251a0b3ea5c" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 45735 } }
I20260807 10:55:31.197926 1406 raft_consensus.cc:374] T f273f8f25c514ec58f7250016dede3f9 P 20ba5901030643c8ae30e251a0b3ea5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:55:31.198079 1406 raft_consensus.cc:729] T f273f8f25c514ec58f7250016dede3f9 P 20ba5901030643c8ae30e251a0b3ea5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20ba5901030643c8ae30e251a0b3ea5c, State: Initialized, Role: FOLLOWER
I20260807 10:55:31.198809 1406 consensus_queue.cc:260] T f273f8f25c514ec58f7250016dede3f9 P 20ba5901030643c8ae30e251a0b3ea5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a08e63525d52423bbbf6911a1573ccde" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 46793 } } peers { permanent_uuid: "1b541bf169b8418ca965c633c309b4a9" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 37639 } } peers { permanent_uuid: "20ba5901030643c8ae30e251a0b3ea5c" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 45735 } }
I20260807 10:55:31.200145 1181 heartbeater.cc:503] Master 127.3.73.254:39409 was elected leader, sending a full tablet report...
I20260807 10:55:31.200775 1406 ts_tablet_manager.cc:1428] T f273f8f25c514ec58f7250016dede3f9 P 20ba5901030643c8ae30e251a0b3ea5c: Time spent starting tablet: real 0.012s user 0.009s sys 0.001s
I20260807 10:55:31.200459 1407 raft_consensus.cc:348] T f273f8f25c514ec58f7250016dede3f9 P a08e63525d52423bbbf6911a1573ccde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a08e63525d52423bbbf6911a1573ccde" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 46793 } } peers { permanent_uuid: "1b541bf169b8418ca965c633c309b4a9" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 37639 } } peers { permanent_uuid: "20ba5901030643c8ae30e251a0b3ea5c" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 45735 } }
I20260807 10:55:31.201043 1407 raft_consensus.cc:374] T f273f8f25c514ec58f7250016dede3f9 P a08e63525d52423bbbf6911a1573ccde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:55:31.201195 1407 raft_consensus.cc:729] T f273f8f25c514ec58f7250016dede3f9 P a08e63525d52423bbbf6911a1573ccde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a08e63525d52423bbbf6911a1573ccde, State: Initialized, Role: FOLLOWER
I20260807 10:55:31.201206 1405 raft_consensus.cc:348] T f273f8f25c514ec58f7250016dede3f9 P 1b541bf169b8418ca965c633c309b4a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a08e63525d52423bbbf6911a1573ccde" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 46793 } } peers { permanent_uuid: "1b541bf169b8418ca965c633c309b4a9" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 37639 } } peers { permanent_uuid: "20ba5901030643c8ae30e251a0b3ea5c" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 45735 } }
I20260807 10:55:31.201730 1405 raft_consensus.cc:374] T f273f8f25c514ec58f7250016dede3f9 P 1b541bf169b8418ca965c633c309b4a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:55:31.201854 1405 raft_consensus.cc:729] T f273f8f25c514ec58f7250016dede3f9 P 1b541bf169b8418ca965c633c309b4a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b541bf169b8418ca965c633c309b4a9, State: Initialized, Role: FOLLOWER
I20260807 10:55:31.201886 1407 consensus_queue.cc:260] T f273f8f25c514ec58f7250016dede3f9 P a08e63525d52423bbbf6911a1573ccde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a08e63525d52423bbbf6911a1573ccde" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 46793 } } peers { permanent_uuid: "1b541bf169b8418ca965c633c309b4a9" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 37639 } } peers { permanent_uuid: "20ba5901030643c8ae30e251a0b3ea5c" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 45735 } }
I20260807 10:55:31.202478 1405 consensus_queue.cc:260] T f273f8f25c514ec58f7250016dede3f9 P 1b541bf169b8418ca965c633c309b4a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a08e63525d52423bbbf6911a1573ccde" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 46793 } } peers { permanent_uuid: "1b541bf169b8418ca965c633c309b4a9" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 37639 } } peers { permanent_uuid: "20ba5901030643c8ae30e251a0b3ea5c" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 45735 } }
I20260807 10:55:31.204175 1407 ts_tablet_manager.cc:1428] T f273f8f25c514ec58f7250016dede3f9 P a08e63525d52423bbbf6911a1573ccde: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260807 10:55:31.205324 1374 heartbeater.cc:503] Master 127.3.73.254:39409 was elected leader, sending a full tablet report...
I20260807 10:55:31.206625 1405 ts_tablet_manager.cc:1428] T f273f8f25c514ec58f7250016dede3f9 P 1b541bf169b8418ca965c633c309b4a9: Time spent starting tablet: real 0.018s user 0.008s sys 0.007s
I20260807 10:55:31.219434 1415 raft_consensus.cc:482] T f273f8f25c514ec58f7250016dede3f9 P a08e63525d52423bbbf6911a1573ccde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 10:55:31.219836 1415 raft_consensus.cc:504] T f273f8f25c514ec58f7250016dede3f9 P a08e63525d52423bbbf6911a1573ccde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a08e63525d52423bbbf6911a1573ccde" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 46793 } } peers { permanent_uuid: "1b541bf169b8418ca965c633c309b4a9" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 37639 } } peers { permanent_uuid: "20ba5901030643c8ae30e251a0b3ea5c" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 45735 } }
I20260807 10:55:31.221776 1415 leader_election.cc:288] T f273f8f25c514ec58f7250016dede3f9 P a08e63525d52423bbbf6911a1573ccde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b541bf169b8418ca965c633c309b4a9 (127.3.73.195:37639), 20ba5901030643c8ae30e251a0b3ea5c (127.3.73.194:45735)
W20260807 10:55:31.227850 988 tablet.cc:2369] T f273f8f25c514ec58f7250016dede3f9 P a08e63525d52423bbbf6911a1573ccde: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:55:31.230556 1398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:31.222401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:46369 (local address 127.3.73.195:37639)
0807 10:55:31.222666 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:31.222677 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:31.223342 (+ 665us) server_negotiation.cc:407] Connection header received
0807 10:55:31.223578 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:31.223586 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:31.223675 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:31.223791 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:55:31.224600 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:31.225735 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:31.228377 (+ 2642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:31.229630 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:31.229760 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:31.229816 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:31.230238 (+ 422us) server_negotiation.cc:299] Negotiation successful
0807 10:55:31.230350 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 10:55:31.230681 1418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:31.222125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.195:37639 (local address 127.3.73.193:46369)
0807 10:55:31.223211 (+ 1086us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:31.223235 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:31.223406 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:31.223839 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:31.223844 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:31.223875 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:55:31.224410 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:31.224427 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:31.225909 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:31.225915 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:31.228138 (+ 2223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:31.228158 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:31.229824 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:31.229831 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:31.230008 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:31.230047 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 10:55:31.230158 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 10:55:31.230312 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":949,"thread_start_us":755,"threads_started":1}
I20260807 10:55:31.231603 1419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:31.223861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.194:45735 (local address 127.3.73.193:37729)
0807 10:55:31.224600 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0807 10:55:31.224622 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 10:55:31.224858 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:55:31.225273 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:55:31.225279 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:55:31.225307 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 10:55:31.225687 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:31.225700 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:31.227219 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:31.227223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:31.229244 (+ 2021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:55:31.229262 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:55:31.231070 (+ 1808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:55:31.231075 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:55:31.231201 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:31.231224 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 10:55:31.231298 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 10:55:31.231394 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":366,"threads_started":1}
I20260807 10:55:31.231791 1404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:31.223657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.73.193:37729 (local address 127.3.73.194:45735)
0807 10:55:31.223901 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:31.223910 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:31.224772 (+ 862us) server_negotiation.cc:407] Connection header received
0807 10:55:31.225060 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:31.225064 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:31.225141 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:31.225226 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 10:55:31.225935 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:31.227048 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:31.229473 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:31.230629 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:31.230793 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:31.230922 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:31.231366 (+ 444us) server_negotiation.cc:299] Negotiation successful
0807 10:55:31.231520 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 10:55:31.231832 1312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f273f8f25c514ec58f7250016dede3f9" candidate_uuid: "a08e63525d52423bbbf6911a1573ccde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b541bf169b8418ca965c633c309b4a9" is_pre_election: true
I20260807 10:55:31.232543 1312 raft_consensus.cc:2454] T f273f8f25c514ec58f7250016dede3f9 P 1b541bf169b8418ca965c633c309b4a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a08e63525d52423bbbf6911a1573ccde in term 0.
I20260807 10:55:31.232997 1115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f273f8f25c514ec58f7250016dede3f9" candidate_uuid: "a08e63525d52423bbbf6911a1573ccde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20ba5901030643c8ae30e251a0b3ea5c" is_pre_election: true
I20260807 10:55:31.233428 831 leader_election.cc:302] T f273f8f25c514ec58f7250016dede3f9 P a08e63525d52423bbbf6911a1573ccde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b541bf169b8418ca965c633c309b4a9, a08e63525d52423bbbf6911a1573ccde; no voters:
I20260807 10:55:31.233522 1115 raft_consensus.cc:2454] T f273f8f25c514ec58f7250016dede3f9 P 20ba5901030643c8ae30e251a0b3ea5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a08e63525d52423bbbf6911a1573ccde in term 0.
I20260807 10:55:31.233891 1415 raft_consensus.cc:2781] T f273f8f25c514ec58f7250016dede3f9 P a08e63525d52423bbbf6911a1573ccde [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 10:55:31.234097 1415 raft_consensus.cc:482] T f273f8f25c514ec58f7250016dede3f9 P a08e63525d52423bbbf6911a1573ccde [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 10:55:31.234236 1415 raft_consensus.cc:3037] T f273f8f25c514ec58f7250016dede3f9 P a08e63525d52423bbbf6911a1573ccde [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:55:31.236584 1415 raft_consensus.cc:504] T f273f8f25c514ec58f7250016dede3f9 P a08e63525d52423bbbf6911a1573ccde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a08e63525d52423bbbf6911a1573ccde" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 46793 } } peers { permanent_uuid: "1b541bf169b8418ca965c633c309b4a9" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 37639 } } peers { permanent_uuid: "20ba5901030643c8ae30e251a0b3ea5c" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 45735 } }
I20260807 10:55:31.237391 1415 leader_election.cc:288] T f273f8f25c514ec58f7250016dede3f9 P a08e63525d52423bbbf6911a1573ccde [CANDIDATE]: Term 1 election: Requested vote from peers 1b541bf169b8418ca965c633c309b4a9 (127.3.73.195:37639), 20ba5901030643c8ae30e251a0b3ea5c (127.3.73.194:45735)
I20260807 10:55:31.238070 1312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f273f8f25c514ec58f7250016dede3f9" candidate_uuid: "a08e63525d52423bbbf6911a1573ccde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b541bf169b8418ca965c633c309b4a9"
I20260807 10:55:31.238119 1115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f273f8f25c514ec58f7250016dede3f9" candidate_uuid: "a08e63525d52423bbbf6911a1573ccde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20ba5901030643c8ae30e251a0b3ea5c"
I20260807 10:55:31.238291 1312 raft_consensus.cc:3037] T f273f8f25c514ec58f7250016dede3f9 P 1b541bf169b8418ca965c633c309b4a9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:55:31.238404 1115 raft_consensus.cc:3037] T f273f8f25c514ec58f7250016dede3f9 P 20ba5901030643c8ae30e251a0b3ea5c [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:55:31.241011 1312 raft_consensus.cc:2454] T f273f8f25c514ec58f7250016dede3f9 P 1b541bf169b8418ca965c633c309b4a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a08e63525d52423bbbf6911a1573ccde in term 1.
I20260807 10:55:31.241062 1115 raft_consensus.cc:2454] T f273f8f25c514ec58f7250016dede3f9 P 20ba5901030643c8ae30e251a0b3ea5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a08e63525d52423bbbf6911a1573ccde in term 1.
I20260807 10:55:31.241626 831 leader_election.cc:302] T f273f8f25c514ec58f7250016dede3f9 P a08e63525d52423bbbf6911a1573ccde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20ba5901030643c8ae30e251a0b3ea5c, a08e63525d52423bbbf6911a1573ccde; no voters:
I20260807 10:55:31.242024 1415 raft_consensus.cc:2781] T f273f8f25c514ec58f7250016dede3f9 P a08e63525d52423bbbf6911a1573ccde [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:55:31.242620 1415 raft_consensus.cc:686] T f273f8f25c514ec58f7250016dede3f9 P a08e63525d52423bbbf6911a1573ccde [term 1 LEADER]: Becoming Leader. State: Replica: a08e63525d52423bbbf6911a1573ccde, State: Running, Role: LEADER
I20260807 10:55:31.243278 1415 consensus_queue.cc:237] T f273f8f25c514ec58f7250016dede3f9 P a08e63525d52423bbbf6911a1573ccde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a08e63525d52423bbbf6911a1573ccde" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 46793 } } peers { permanent_uuid: "1b541bf169b8418ca965c633c309b4a9" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 37639 } } peers { permanent_uuid: "20ba5901030643c8ae30e251a0b3ea5c" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 45735 } }
I20260807 10:55:31.249056 714 catalog_manager.cc:5495] T f273f8f25c514ec58f7250016dede3f9 P a08e63525d52423bbbf6911a1573ccde reported cstate change: term changed from 0 to 1, leader changed from <none> to a08e63525d52423bbbf6911a1573ccde (127.3.73.193). New cstate: current_term: 1 leader_uuid: "a08e63525d52423bbbf6911a1573ccde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a08e63525d52423bbbf6911a1573ccde" member_type: VOTER last_known_addr { host: "127.3.73.193" port: 46793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b541bf169b8418ca965c633c309b4a9" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 37639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20ba5901030643c8ae30e251a0b3ea5c" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 45735 } health_report { overall_health: UNKNOWN } } }
I20260807 10:55:31.283655 1396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:31.276900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35560 (local address 127.3.73.193:46793)
0807 10:55:31.277316 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:31.277321 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:31.277340 (+ 19us) server_negotiation.cc:407] Connection header received
0807 10:55:31.277444 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:31.277447 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:31.277563 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:31.277706 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:55:31.278251 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:31.279036 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:31.281204 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:31.281746 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:31.281878 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:31.282478 (+ 600us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:55:31.283092 (+ 614us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:55:31.283215 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:31.283366 (+ 151us) server_negotiation.cc:299] Negotiation successful
0807 10:55:31.283444 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233}
I20260807 10:55:31.293349 1115 raft_consensus.cc:1261] T f273f8f25c514ec58f7250016dede3f9 P 20ba5901030643c8ae30e251a0b3ea5c [term 1 FOLLOWER]: Refusing update from remote peer a08e63525d52423bbbf6911a1573ccde: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:55:31.293347 1312 raft_consensus.cc:1261] T f273f8f25c514ec58f7250016dede3f9 P 1b541bf169b8418ca965c633c309b4a9 [term 1 FOLLOWER]: Refusing update from remote peer a08e63525d52423bbbf6911a1573ccde: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:55:31.294514 1420 consensus_queue.cc:1035] T f273f8f25c514ec58f7250016dede3f9 P a08e63525d52423bbbf6911a1573ccde [LEADER]: Connected to new peer: Peer: permanent_uuid: "20ba5901030643c8ae30e251a0b3ea5c" member_type: VOTER last_known_addr { host: "127.3.73.194" port: 45735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 10:55:31.295053 1182 tablet.cc:2369] T f273f8f25c514ec58f7250016dede3f9 P 20ba5901030643c8ae30e251a0b3ea5c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:55:31.295135 1415 consensus_queue.cc:1035] T f273f8f25c514ec58f7250016dede3f9 P a08e63525d52423bbbf6911a1573ccde [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b541bf169b8418ca965c633c309b4a9" member_type: VOTER last_known_addr { host: "127.3.73.195" port: 37639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:55:31.308852 1427 mvcc.cc:205] Tried to move back new op lower bound from 7315866137765126144 to 7315866137577156608. Current Snapshot: MvccSnapshot[applied={T|T < 7315866137765126144}]
I20260807 10:55:31.310475 1428 mvcc.cc:205] Tried to move back new op lower bound from 7315866137765126144 to 7315866137577156608. Current Snapshot: MvccSnapshot[applied={T|T < 7315866137765126144}]
I20260807 10:55:31.326184 714 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59058:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:55:31.326537 714 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59058:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 10:55:31.331202 714 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 63c715a1f0254c5d8e88ea1cbf12c2b5: Sending DeleteTablet for 3 replicas of tablet f273f8f25c514ec58f7250016dede3f9
I20260807 10:55:31.333196 1083 tablet_service.cc:1514] Processing DeleteTablet for tablet f273f8f25c514ec58f7250016dede3f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:55:31 UTC) from {username='slave'} at 127.0.0.1:46904
I20260807 10:55:31.333779 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 804
I20260807 10:55:31.334271 1281 tablet_service.cc:1514] Processing DeleteTablet for tablet f273f8f25c514ec58f7250016dede3f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:55:31 UTC) from {username='slave'} at 127.0.0.1:55552
I20260807 10:55:31.335270 896 tablet_service.cc:1514] Processing DeleteTablet for tablet f273f8f25c514ec58f7250016dede3f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 10:55:31 UTC) from {username='slave'} at 127.0.0.1:35558
I20260807 10:55:31.349326 1440 tablet_replica.cc:330] T f273f8f25c514ec58f7250016dede3f9 P 20ba5901030643c8ae30e251a0b3ea5c: stopping tablet replica
I20260807 10:55:31.349931 1440 raft_consensus.cc:2229] T f273f8f25c514ec58f7250016dede3f9 P 20ba5901030643c8ae30e251a0b3ea5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:55:31.351090 1440 raft_consensus.cc:2258] T f273f8f25c514ec58f7250016dede3f9 P 20ba5901030643c8ae30e251a0b3ea5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:55:31.353587 1442 tablet_replica.cc:330] T f273f8f25c514ec58f7250016dede3f9 P 1b541bf169b8418ca965c633c309b4a9: stopping tablet replica
I20260807 10:55:31.354395 1442 raft_consensus.cc:2229] T f273f8f25c514ec58f7250016dede3f9 P 1b541bf169b8418ca965c633c309b4a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 10:55:31.355039 1444 tablet_replica.cc:330] T f273f8f25c514ec58f7250016dede3f9 P a08e63525d52423bbbf6911a1573ccde: stopping tablet replica
I20260807 10:55:31.355103 1442 raft_consensus.cc:2258] T f273f8f25c514ec58f7250016dede3f9 P 1b541bf169b8418ca965c633c309b4a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:55:31.355819 1444 raft_consensus.cc:2229] T f273f8f25c514ec58f7250016dede3f9 P a08e63525d52423bbbf6911a1573ccde [term 1 LEADER]: Raft consensus shutting down.
I20260807 10:55:31.356655 1444 raft_consensus.cc:2258] T f273f8f25c514ec58f7250016dede3f9 P a08e63525d52423bbbf6911a1573ccde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 10:55:31.357792 1442 ts_tablet_manager.cc:1905] T f273f8f25c514ec58f7250016dede3f9 P 1b541bf169b8418ca965c633c309b4a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:55:31.357780 1440 ts_tablet_manager.cc:1905] T f273f8f25c514ec58f7250016dede3f9 P 20ba5901030643c8ae30e251a0b3ea5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:55:31.358753 1444 ts_tablet_manager.cc:1905] T f273f8f25c514ec58f7250016dede3f9 P a08e63525d52423bbbf6911a1573ccde: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 10:55:31.366029 1442 ts_tablet_manager.cc:1918] T f273f8f25c514ec58f7250016dede3f9 P 1b541bf169b8418ca965c633c309b4a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:55:31.366029 1440 ts_tablet_manager.cc:1918] T f273f8f25c514ec58f7250016dede3f9 P 20ba5901030643c8ae30e251a0b3ea5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:55:31.366292 1440 log.cc:1196] T f273f8f25c514ec58f7250016dede3f9 P 20ba5901030643c8ae30e251a0b3ea5c: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-1/wal/wals/f273f8f25c514ec58f7250016dede3f9
I20260807 10:55:31.366405 1442 log.cc:1196] T f273f8f25c514ec58f7250016dede3f9 P 1b541bf169b8418ca965c633c309b4a9: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-2/wal/wals/f273f8f25c514ec58f7250016dede3f9
I20260807 10:55:31.367169 1442 ts_tablet_manager.cc:1939] T f273f8f25c514ec58f7250016dede3f9 P 1b541bf169b8418ca965c633c309b4a9: Deleting consensus metadata
I20260807 10:55:31.368605 1444 ts_tablet_manager.cc:1918] T f273f8f25c514ec58f7250016dede3f9 P a08e63525d52423bbbf6911a1573ccde: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 10:55:31.368950 1444 log.cc:1196] T f273f8f25c514ec58f7250016dede3f9 P a08e63525d52423bbbf6911a1573ccde: Deleting WAL directory at /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786099767749326-3367-0/minicluster-data/ts-0/wal/wals/f273f8f25c514ec58f7250016dede3f9
I20260807 10:55:31.369666 1444 ts_tablet_manager.cc:1939] T f273f8f25c514ec58f7250016dede3f9 P a08e63525d52423bbbf6911a1573ccde: Deleting consensus metadata
I20260807 10:55:31.369807 701 catalog_manager.cc:4841] TS 1b541bf169b8418ca965c633c309b4a9 (127.3.73.195:37639): tablet f273f8f25c514ec58f7250016dede3f9 (table test-table [id=7b7c92686c864e13aeed03501b1289fc]) successfully deleted
I20260807 10:55:31.370922 1440 ts_tablet_manager.cc:1939] T f273f8f25c514ec58f7250016dede3f9 P 20ba5901030643c8ae30e251a0b3ea5c: Deleting consensus metadata
I20260807 10:55:31.371870 698 catalog_manager.cc:4841] TS a08e63525d52423bbbf6911a1573ccde (127.3.73.193:46793): tablet f273f8f25c514ec58f7250016dede3f9 (table test-table [id=7b7c92686c864e13aeed03501b1289fc]) successfully deleted
I20260807 10:55:31.373167 1396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:31.335266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35574 (local address 127.3.73.193:46793)
0807 10:55:31.335482 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:31.335490 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:31.346466 (+ 10976us) server_negotiation.cc:407] Connection header received
0807 10:55:31.347641 (+ 1175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:31.347648 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:31.347777 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:31.347874 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:31.348946 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:31.354386 (+ 5440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:31.367977 (+ 13591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:31.368937 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:31.369079 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:31.370988 (+ 1909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:31.371006 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:31.371031 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:31.372121 (+ 1090us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:31.372209 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:31.372241 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:31.372382 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:31.372592 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 10:55:31.372744 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 10:55:31.373129 701 catalog_manager.cc:4841] TS 20ba5901030643c8ae30e251a0b3ea5c (127.3.73.194:45735): tablet f273f8f25c514ec58f7250016dede3f9 (table test-table [id=7b7c92686c864e13aeed03501b1289fc]) successfully deleted
I20260807 10:55:31.374277 978 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:55:31.482199 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 804
I20260807 10:55:31.510723 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 993
I20260807 10:55:31.518606 1404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:31.511675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46916 (local address 127.3.73.194:45735)
0807 10:55:31.511920 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:31.511925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:31.512030 (+ 105us) server_negotiation.cc:407] Connection header received
0807 10:55:31.512264 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:31.512269 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:31.512386 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:31.512500 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:31.512996 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:31.513830 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:31.515254 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:31.515711 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:31.515826 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:31.517096 (+ 1270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:31.517105 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:31.517121 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:31.517763 (+ 642us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:31.517814 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:31.517870 (+ 56us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:31.517967 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:31.518114 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 10:55:31.518210 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 10:55:31.519286 1173 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:55:31.611970 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 993
I20260807 10:55:31.636180 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 1185
I20260807 10:55:31.643603 1398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:31.637027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55568 (local address 127.3.73.195:37639)
0807 10:55:31.637261 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:31.637270 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:31.637294 (+ 24us) server_negotiation.cc:407] Connection header received
0807 10:55:31.637505 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:31.637515 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:31.637608 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:31.637722 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:31.638143 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:31.638888 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:31.640148 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:31.640625 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:31.641025 (+ 400us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:31.642024 (+ 999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:31.642035 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:31.642053 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:31.642681 (+ 628us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:31.642787 (+ 106us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:31.642811 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:31.642958 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:31.643113 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 10:55:31.643242 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 10:55:31.644431 1368 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:55:31.740940 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 1185
I20260807 10:55:31.771441 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 677
I20260807 10:55:31.773097 759 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:55:31.887307 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 677
2026-08-07T10:55:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3164 ms)
[----------] 6 tests from EncryptionPolicyTest (26639 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:55:32 dist-test-slave-x5fp krb5kdc[1452](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:55:32 dist-test-slave-x5fp krb5kdc[1452](info): set up 2 sockets
Aug 07 10:55:32 dist-test-slave-x5fp krb5kdc[1452](info): commencing operation
krb5kdc: starting...
W20260807 10:55:33.286684 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.349s user 0.000s sys 0.063s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:55:33 dist-test-slave-x5fp krb5kdc[1452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:55:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:55:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 10:55:33.693450 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39845
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39845
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:55:34.081977 1467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:55:34.082332 1467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:55:34.082518 1467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:55:34.090474 1467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:55:34.090628 1467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:55:34.090715 1467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:55:34.090814 1467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:55:34.090883 1467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:55:34.102041 1467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39845
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:55:34.103559 1467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:55:34.105234 1467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:55:34.118453 1474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:34.118402 1478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:34.119042 1475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:34.119992 1467 server_base.cc:1034] running on GCE node
I20260807 10:55:34.121006 1467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:55:34.122668 1467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:55:34.123996 1467 hybrid_clock.cc:648] HybridClock initialized: now 1786100134123897 us; error 92 us; skew 500 ppm
I20260807 10:55:34.124442 1467 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:55:34 dist-test-slave-x5fp krb5kdc[1452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100134, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:55:34.134429 1467 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 10:55:34.136302 1467 webserver.cc:460] Webserver started at http://127.0.0.1:44175/ using document root <none> and password file <none>
I20260807 10:55:34.137207 1467 fs_manager.cc:359] Metadata directory not provided
I20260807 10:55:34.137387 1467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:55:34.137720 1467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:55:34.141057 1467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "dcf65d87f42f4d129c339d3d92f4f152"
format_stamp: "Formatted at 2026-08-07 10:55:34 on dist-test-slave-x5fp"
server_key: "462c117cf09552d333a413706af5e3d7"
server_key_iv: "cc59e2feadfadface04d50573c710596"
server_key_version: "encryptionkey@0"
I20260807 10:55:34.141928 1467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "dcf65d87f42f4d129c339d3d92f4f152"
format_stamp: "Formatted at 2026-08-07 10:55:34 on dist-test-slave-x5fp"
server_key: "462c117cf09552d333a413706af5e3d7"
server_key_iv: "cc59e2feadfadface04d50573c710596"
server_key_version: "encryptionkey@0"
I20260807 10:55:34.148212 1467 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 10:55:34.152504 1486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:34.154150 1467 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 10:55:34.154374 1467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "dcf65d87f42f4d129c339d3d92f4f152"
format_stamp: "Formatted at 2026-08-07 10:55:34 on dist-test-slave-x5fp"
server_key: "462c117cf09552d333a413706af5e3d7"
server_key_iv: "cc59e2feadfadface04d50573c710596"
server_key_version: "encryptionkey@0"
I20260807 10:55:34.154624 1467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:55:34.178004 1467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:55:34.183034 1467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:55:34.183359 1467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:55:34.204284 1467 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39845
I20260807 10:55:34.204324 1564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39845 every 8 connection(s)
I20260807 10:55:34.205816 1467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:55:34.207585 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 1467
I20260807 10:55:34.207899 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:55:34.208910 3367 external_mini_cluster.cc:1428] Setting key 6c063b56dabf78f9198e395a40dfc9fd
I20260807 10:55:34.211171 1566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 10:55:34 dist-test-slave-x5fp krb5kdc[1452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 10:55:34.227124 1566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcf65d87f42f4d129c339d3d92f4f152: Bootstrap starting.
I20260807 10:55:34.230744 1566 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dcf65d87f42f4d129c339d3d92f4f152: Neither blocks nor log segments found. Creating new log.
I20260807 10:55:34.232120 1566 log.cc:824] T 00000000000000000000000000000000 P dcf65d87f42f4d129c339d3d92f4f152: Log is configured to *not* fsync() on all Append() calls
I20260807 10:55:34.235703 1566 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dcf65d87f42f4d129c339d3d92f4f152: No bootstrap required, opened a new log
I20260807 10:55:34.236407 1569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:34.211111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60011 (local address 127.0.0.1:39845)
0807 10:55:34.212074 (+ 963us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:34.212090 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:34.212147 (+ 57us) server_negotiation.cc:407] Connection header received
0807 10:55:34.214732 (+ 2585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:34.214766 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:34.215402 (+ 636us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:34.215927 (+ 525us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:34.217489 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:34.219732 (+ 2243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:34.221260 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:34.222461 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:34.222683 (+ 222us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:34.227836 (+ 5153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:34.227846 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:34.227862 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:55:34.227908 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:34.231281 (+ 3373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:34.232455 (+ 1174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:34.232462 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:34.232468 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:34.232707 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:34.233323 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:34.233329 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:34.233332 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:34.234022 (+ 690us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:34.234258 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:34.234896 (+ 638us) server_negotiation.cc:299] Negotiation successful
0807 10:55:34.235233 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":695,"thread_start_us":395,"threads_started":1}
I20260807 10:55:34.241187 3367 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 10:55:34.246809 1566 raft_consensus.cc:348] T 00000000000000000000000000000000 P dcf65d87f42f4d129c339d3d92f4f152 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf65d87f42f4d129c339d3d92f4f152" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39845 } }
I20260807 10:55:34.247386 1566 raft_consensus.cc:374] T 00000000000000000000000000000000 P dcf65d87f42f4d129c339d3d92f4f152 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:55:34.247540 1566 raft_consensus.cc:729] T 00000000000000000000000000000000 P dcf65d87f42f4d129c339d3d92f4f152 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcf65d87f42f4d129c339d3d92f4f152, State: Initialized, Role: FOLLOWER
I20260807 10:55:34.248436 1566 consensus_queue.cc:260] T 00000000000000000000000000000000 P dcf65d87f42f4d129c339d3d92f4f152 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf65d87f42f4d129c339d3d92f4f152" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39845 } }
I20260807 10:55:34.248840 1566 raft_consensus.cc:388] T 00000000000000000000000000000000 P dcf65d87f42f4d129c339d3d92f4f152 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:55:34.249089 1566 raft_consensus.cc:482] T 00000000000000000000000000000000 P dcf65d87f42f4d129c339d3d92f4f152 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:55:34.249368 1566 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dcf65d87f42f4d129c339d3d92f4f152 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:55:34.252669 1566 raft_consensus.cc:504] T 00000000000000000000000000000000 P dcf65d87f42f4d129c339d3d92f4f152 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf65d87f42f4d129c339d3d92f4f152" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39845 } }
I20260807 10:55:34.253393 1566 leader_election.cc:302] T 00000000000000000000000000000000 P dcf65d87f42f4d129c339d3d92f4f152 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dcf65d87f42f4d129c339d3d92f4f152; no voters:
I20260807 10:55:34.254103 1566 leader_election.cc:288] T 00000000000000000000000000000000 P dcf65d87f42f4d129c339d3d92f4f152 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:55:34.254323 1579 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dcf65d87f42f4d129c339d3d92f4f152 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:55:34.255721 1579 raft_consensus.cc:686] T 00000000000000000000000000000000 P dcf65d87f42f4d129c339d3d92f4f152 [term 1 LEADER]: Becoming Leader. State: Replica: dcf65d87f42f4d129c339d3d92f4f152, State: Running, Role: LEADER
I20260807 10:55:34.256551 1579 consensus_queue.cc:237] T 00000000000000000000000000000000 P dcf65d87f42f4d129c339d3d92f4f152 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf65d87f42f4d129c339d3d92f4f152" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39845 } }
I20260807 10:55:34.256776 1566 sys_catalog.cc:564] T 00000000000000000000000000000000 P dcf65d87f42f4d129c339d3d92f4f152 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:55:34.260818 1580 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcf65d87f42f4d129c339d3d92f4f152 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dcf65d87f42f4d129c339d3d92f4f152" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf65d87f42f4d129c339d3d92f4f152" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39845 } } }
I20260807 10:55:34.261864 1580 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcf65d87f42f4d129c339d3d92f4f152 [sys.catalog]: This master's current role is: LEADER
I20260807 10:55:34.262971 1583 sys_catalog.cc:455] T 00000000000000000000000000000000 P dcf65d87f42f4d129c339d3d92f4f152 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dcf65d87f42f4d129c339d3d92f4f152. Latest consensus state: current_term: 1 leader_uuid: "dcf65d87f42f4d129c339d3d92f4f152" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dcf65d87f42f4d129c339d3d92f4f152" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39845 } } }
I20260807 10:55:34.263411 1583 sys_catalog.cc:458] T 00000000000000000000000000000000 P dcf65d87f42f4d129c339d3d92f4f152 [sys.catalog]: This master's current role is: LEADER
I20260807 10:55:34.266732 1592 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:55:34.270509 1569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:34.247608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46901 (local address 127.0.0.1:39845)
0807 10:55:34.248000 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:34.248011 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:34.248603 (+ 592us) server_negotiation.cc:407] Connection header received
0807 10:55:34.248945 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:34.248951 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:34.249058 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:34.249220 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:34.251035 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:34.252357 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:34.254325 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:34.255345 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:34.255537 (+ 192us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:34.260012 (+ 4475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:34.260020 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:34.260026 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:55:34.260076 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:34.265638 (+ 5562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:34.267241 (+ 1603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:34.267248 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:34.267253 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:34.267496 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:34.268362 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:34.268369 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:34.268374 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:34.268890 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:34.269118 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:34.269728 (+ 610us) server_negotiation.cc:299] Negotiation successful
0807 10:55:34.269944 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191}
I20260807 10:55:34.274995 1592 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:55:34.287320 1592 catalog_manager.cc:1347] Generated new cluster ID: 41217767b5d94c1ebaa3e03d4b6dda7b
I20260807 10:55:34.287492 1592 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:55:34.319542 1592 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:55:34.321048 1592 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:55:34.334344 1592 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dcf65d87f42f4d129c339d3d92f4f152: Generated new TSK 0
I20260807 10:55:34.335326 1592 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 10:55:34.343472 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 1467
I20260807 10:55:34.345736 1555 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:55:34.441916 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 1467
W20260807 10:55:34.470168 1449 connection.cc:535] client connection to 127.0.0.1:39845 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39845): Connection reset by peer (error 104)
2026-08-07T10:55:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2553 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:55:34 dist-test-slave-x5fp krb5kdc[1609](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:55:34 dist-test-slave-x5fp krb5kdc[1609](info): set up 2 sockets
Aug 07 10:55:34 dist-test-slave-x5fp krb5kdc[1609](info): commencing operation
krb5kdc: starting...
W20260807 10:55:35.827324 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.337s user 0.000s sys 0.062s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:55:36 dist-test-slave-x5fp krb5kdc[1609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:55:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:55:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 10:55:36.228967 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40115
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40115
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:55:36.617511 1630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:55:36.617913 1630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:55:36.618090 1630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:55:36.625809 1630 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:55:36.626014 1630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:55:36.626127 1630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:55:36.626250 1630 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:55:36.626328 1630 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:55:36.636965 1630 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40115
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:55:36.638440 1630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:55:36.639886 1630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:55:36.653091 1643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:36.652957 1637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:36.653146 1639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:36.653825 1630 server_base.cc:1034] running on GCE node
I20260807 10:55:36.654764 1630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:55:36.656445 1630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:55:36.657689 1630 hybrid_clock.cc:648] HybridClock initialized: now 1786100136657629 us; error 78 us; skew 500 ppm
I20260807 10:55:36.658069 1630 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:55:36 dist-test-slave-x5fp krb5kdc[1609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100136, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:55:36.661672 1630 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 10:55:36.663479 1630 webserver.cc:460] Webserver started at http://127.0.0.1:33319/ using document root <none> and password file <none>
I20260807 10:55:36.664250 1630 fs_manager.cc:359] Metadata directory not provided
I20260807 10:55:36.664417 1630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:55:36.664839 1630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:55:36.668166 1630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "6af2d9db024640559dc0172ed381ecc2"
format_stamp: "Formatted at 2026-08-07 10:55:36 on dist-test-slave-x5fp"
server_key: "2a42235ea86cbb20d91f4967d75e198e"
server_key_iv: "b13f0bf1723872644eee71ccbde11e04"
server_key_version: "encryptionkey@0"
I20260807 10:55:36.669071 1630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "6af2d9db024640559dc0172ed381ecc2"
format_stamp: "Formatted at 2026-08-07 10:55:36 on dist-test-slave-x5fp"
server_key: "2a42235ea86cbb20d91f4967d75e198e"
server_key_iv: "b13f0bf1723872644eee71ccbde11e04"
server_key_version: "encryptionkey@0"
I20260807 10:55:36.674821 1630 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 10:55:36.678877 1653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:36.680369 1630 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 10:55:36.680608 1630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "6af2d9db024640559dc0172ed381ecc2"
format_stamp: "Formatted at 2026-08-07 10:55:36 on dist-test-slave-x5fp"
server_key: "2a42235ea86cbb20d91f4967d75e198e"
server_key_iv: "b13f0bf1723872644eee71ccbde11e04"
server_key_version: "encryptionkey@0"
I20260807 10:55:36.680918 1630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:55:36.701821 1630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:55:36.706528 1630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:55:36.706874 1630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:55:36.727471 1630 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40115
I20260807 10:55:36.727471 1735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40115 every 8 connection(s)
I20260807 10:55:36.729262 1630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:55:36.730379 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 1630
I20260807 10:55:36.730581 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:55:36.731011 3367 external_mini_cluster.cc:1428] Setting key 006809748246910af335634dfd7433a4
I20260807 10:55:36.734506 1736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:55:36.748096 1739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:36.732951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60863 (local address 127.0.0.1:40115)
0807 10:55:36.733893 (+ 942us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:36.733910 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:36.733965 (+ 55us) server_negotiation.cc:407] Connection header received
0807 10:55:36.736436 (+ 2471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:36.736486 (+ 50us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:36.737084 (+ 598us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:36.737586 (+ 502us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:36.738186 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:36.740314 (+ 2128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:36.742072 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:36.743180 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:36.743389 (+ 209us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:36.744862 (+ 1473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:36.744876 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:36.744901 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:36.745721 (+ 820us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:36.745831 (+ 110us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:36.745872 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:36.746061 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:36.746368 (+ 307us) server_negotiation.cc:299] Negotiation successful
0807 10:55:36.746589 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":687,"thread_start_us":412,"threads_started":1}
I20260807 10:55:36.749974 1736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6af2d9db024640559dc0172ed381ecc2: Bootstrap starting.
I20260807 10:55:36.753228 3367 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 10:55:36.753963 1736 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6af2d9db024640559dc0172ed381ecc2: Neither blocks nor log segments found. Creating new log.
I20260807 10:55:36.755054 1736 log.cc:824] T 00000000000000000000000000000000 P 6af2d9db024640559dc0172ed381ecc2: Log is configured to *not* fsync() on all Append() calls
I20260807 10:55:36.759037 1736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6af2d9db024640559dc0172ed381ecc2: No bootstrap required, opened a new log
I20260807 10:55:36.765811 1739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:36.758471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51459 (local address 127.0.0.1:40115)
0807 10:55:36.758711 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:36.758720 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:36.759147 (+ 427us) server_negotiation.cc:407] Connection header received
0807 10:55:36.759414 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:36.759418 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:36.759496 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:36.759633 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:36.760185 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:36.761204 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:36.762782 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:36.763336 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:36.763514 (+ 178us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:36.764902 (+ 1388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:36.764908 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:36.764911 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:36.764934 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:36.764984 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:36.765002 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:36.765157 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:36.765399 (+ 242us) server_negotiation.cc:299] Negotiation successful
0807 10:55:36.765471 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 10:55:36.768122 1736 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6af2d9db024640559dc0172ed381ecc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af2d9db024640559dc0172ed381ecc2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40115 } }
I20260807 10:55:36.768555 1736 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6af2d9db024640559dc0172ed381ecc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:55:36.768666 1736 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6af2d9db024640559dc0172ed381ecc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6af2d9db024640559dc0172ed381ecc2, State: Initialized, Role: FOLLOWER
I20260807 10:55:36.769362 1736 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6af2d9db024640559dc0172ed381ecc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af2d9db024640559dc0172ed381ecc2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40115 } }
I20260807 10:55:36.769626 1736 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6af2d9db024640559dc0172ed381ecc2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:55:36.769780 1736 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6af2d9db024640559dc0172ed381ecc2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:55:36.769979 1736 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6af2d9db024640559dc0172ed381ecc2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:55:36.772508 1736 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6af2d9db024640559dc0172ed381ecc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af2d9db024640559dc0172ed381ecc2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40115 } }
I20260807 10:55:36.773221 1736 leader_election.cc:302] T 00000000000000000000000000000000 P 6af2d9db024640559dc0172ed381ecc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6af2d9db024640559dc0172ed381ecc2; no voters:
I20260807 10:55:36.773902 1736 leader_election.cc:288] T 00000000000000000000000000000000 P 6af2d9db024640559dc0172ed381ecc2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:55:36.774124 1750 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6af2d9db024640559dc0172ed381ecc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:55:36.774941 1750 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6af2d9db024640559dc0172ed381ecc2 [term 1 LEADER]: Becoming Leader. State: Replica: 6af2d9db024640559dc0172ed381ecc2, State: Running, Role: LEADER
I20260807 10:55:36.775548 1750 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6af2d9db024640559dc0172ed381ecc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af2d9db024640559dc0172ed381ecc2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40115 } }
I20260807 10:55:36.776228 1736 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6af2d9db024640559dc0172ed381ecc2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:55:36.778843 1751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6af2d9db024640559dc0172ed381ecc2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6af2d9db024640559dc0172ed381ecc2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af2d9db024640559dc0172ed381ecc2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40115 } } }
I20260807 10:55:36.779325 1751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6af2d9db024640559dc0172ed381ecc2 [sys.catalog]: This master's current role is: LEADER
I20260807 10:55:36.779563 1752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6af2d9db024640559dc0172ed381ecc2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6af2d9db024640559dc0172ed381ecc2. Latest consensus state: current_term: 1 leader_uuid: "6af2d9db024640559dc0172ed381ecc2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6af2d9db024640559dc0172ed381ecc2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40115 } } }
I20260807 10:55:36.780104 1752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6af2d9db024640559dc0172ed381ecc2 [sys.catalog]: This master's current role is: LEADER
I20260807 10:55:36.782675 1757 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:55:36.788866 1757 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:55:36.799938 1757 catalog_manager.cc:1347] Generated new cluster ID: 9553095d6ef646a698877a2714bfac3d
I20260807 10:55:36.800108 1757 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:55:36.812227 1757 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:55:36.814316 1757 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:55:36.835491 1757 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6af2d9db024640559dc0172ed381ecc2: Generated new TSK 0
I20260807 10:55:36.836511 1757 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 10:55:36.843255 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 1630
W20260807 10:55:36.844553 1726 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:60863
W20260807 10:55:36.845505 3367 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 1630: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 10:55:36.845723 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 1630
W20260807 10:55:36.870285 1606 connection.cc:535] client connection to 127.0.0.1:40115 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40115): Connection reset by peer (error 104)
2026-08-07T10:55:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2399 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:55:36 dist-test-slave-x5fp krb5kdc[1776](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:55:36 dist-test-slave-x5fp krb5kdc[1776](info): set up 2 sockets
Aug 07 10:55:36 dist-test-slave-x5fp krb5kdc[1776](info): commencing operation
krb5kdc: starting...
W20260807 10:55:38.270859 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.381s user 0.001s sys 0.060s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:55:38 dist-test-slave-x5fp krb5kdc[1776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:55:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:55:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 10:55:38.692827 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40019
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40019
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:55:39.083178 1792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:55:39.083525 1792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:55:39.083626 1792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:55:39.091488 1792 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:55:39.091704 1792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:55:39.091780 1792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:55:39.091929 1792 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:55:39.092039 1792 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:55:39.103467 1792 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40019
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:55:39.104915 1792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:55:39.106298 1792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:55:39.119926 1801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:39.119977 1800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:39.120131 1803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:39.120641 1792 server_base.cc:1034] running on GCE node
I20260807 10:55:39.121675 1792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:55:39.123405 1792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:55:39.124642 1792 hybrid_clock.cc:648] HybridClock initialized: now 1786100139124599 us; error 33 us; skew 500 ppm
I20260807 10:55:39.125110 1792 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:55:39 dist-test-slave-x5fp krb5kdc[1776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100139, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:55:39.129060 1792 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 10:55:39.131069 1792 webserver.cc:460] Webserver started at http://127.0.0.1:39109/ using document root <none> and password file <none>
I20260807 10:55:39.131896 1792 fs_manager.cc:359] Metadata directory not provided
I20260807 10:55:39.132068 1792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:55:39.132459 1792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:55:39.135963 1792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "ef64bb1caba642ec8cf0f246e313a71d"
format_stamp: "Formatted at 2026-08-07 10:55:39 on dist-test-slave-x5fp"
server_key: "39f8e3c6061f5710a52072d90ee524e8"
server_key_iv: "84b37aa9d708a8a7db9cd146ec42a7dc"
server_key_version: "encryptionkey@0"
I20260807 10:55:39.136965 1792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "ef64bb1caba642ec8cf0f246e313a71d"
format_stamp: "Formatted at 2026-08-07 10:55:39 on dist-test-slave-x5fp"
server_key: "39f8e3c6061f5710a52072d90ee524e8"
server_key_iv: "84b37aa9d708a8a7db9cd146ec42a7dc"
server_key_version: "encryptionkey@0"
I20260807 10:55:39.143111 1792 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 10:55:39.147526 1815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:39.149411 1792 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 10:55:39.149677 1792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "ef64bb1caba642ec8cf0f246e313a71d"
format_stamp: "Formatted at 2026-08-07 10:55:39 on dist-test-slave-x5fp"
server_key: "39f8e3c6061f5710a52072d90ee524e8"
server_key_iv: "84b37aa9d708a8a7db9cd146ec42a7dc"
server_key_version: "encryptionkey@0"
I20260807 10:55:39.149991 1792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:55:39.157279 1792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:55:39.172255 1792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:55:39.172648 1792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:55:39.193663 1792 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40019
I20260807 10:55:39.193787 1897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40019 every 8 connection(s)
I20260807 10:55:39.195254 1792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:55:39.199654 1898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:55:39.204823 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 1792
I20260807 10:55:39.205042 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:55:39.205684 3367 external_mini_cluster.cc:1428] Setting key 13d2c9ec2c357d3a8f0a58f324cf0ec2
I20260807 10:55:39.213346 1898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef64bb1caba642ec8cf0f246e313a71d: Bootstrap starting.
I20260807 10:55:39.215462 1902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:39.207839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43703 (local address 127.0.0.1:40019)
0807 10:55:39.208893 (+ 1054us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:39.208913 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:39.208972 (+ 59us) server_negotiation.cc:407] Connection header received
0807 10:55:39.211336 (+ 2364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:39.211367 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:39.211780 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:39.212217 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:39.213674 (+ 1457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:39.213681 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:39.213696 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:39.213713 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:39.213790 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:39.213813 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:39.214090 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:39.214201 (+ 111us) server_negotiation.cc:299] Negotiation successful
0807 10:55:39.214419 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":398,"threads_started":1}
I20260807 10:55:39.217324 1898 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef64bb1caba642ec8cf0f246e313a71d: Neither blocks nor log segments found. Creating new log.
I20260807 10:55:39.218722 1898 log.cc:824] T 00000000000000000000000000000000 P ef64bb1caba642ec8cf0f246e313a71d: Log is configured to *not* fsync() on all Append() calls
I20260807 10:55:39.219483 3367 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 10:55:39.222169 1898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef64bb1caba642ec8cf0f246e313a71d: No bootstrap required, opened a new log
I20260807 10:55:39.227612 1902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:39.224470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39461 (local address 127.0.0.1:40019)
0807 10:55:39.224651 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:39.224656 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:39.225139 (+ 483us) server_negotiation.cc:407] Connection header received
0807 10:55:39.225391 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:39.225396 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:39.225471 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:39.225581 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:39.226760 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:39.226765 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:39.226768 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:39.226788 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:39.226833 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:39.226850 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:39.226994 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:39.227169 (+ 175us) server_negotiation.cc:299] Negotiation successful
0807 10:55:39.227240 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 10:55:39.232071 1898 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef64bb1caba642ec8cf0f246e313a71d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef64bb1caba642ec8cf0f246e313a71d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40019 } }
I20260807 10:55:39.232654 1898 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef64bb1caba642ec8cf0f246e313a71d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:55:39.232837 1898 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef64bb1caba642ec8cf0f246e313a71d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef64bb1caba642ec8cf0f246e313a71d, State: Initialized, Role: FOLLOWER
I20260807 10:55:39.233534 1898 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef64bb1caba642ec8cf0f246e313a71d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef64bb1caba642ec8cf0f246e313a71d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40019 } }
I20260807 10:55:39.233873 1898 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef64bb1caba642ec8cf0f246e313a71d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:55:39.234205 1898 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef64bb1caba642ec8cf0f246e313a71d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:55:39.234458 1898 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef64bb1caba642ec8cf0f246e313a71d [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:55:39.237416 1898 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef64bb1caba642ec8cf0f246e313a71d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef64bb1caba642ec8cf0f246e313a71d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40019 } }
I20260807 10:55:39.238108 1898 leader_election.cc:302] T 00000000000000000000000000000000 P ef64bb1caba642ec8cf0f246e313a71d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef64bb1caba642ec8cf0f246e313a71d; no voters:
I20260807 10:55:39.238801 1898 leader_election.cc:288] T 00000000000000000000000000000000 P ef64bb1caba642ec8cf0f246e313a71d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:55:39.238973 1910 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef64bb1caba642ec8cf0f246e313a71d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:55:39.240270 1910 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef64bb1caba642ec8cf0f246e313a71d [term 1 LEADER]: Becoming Leader. State: Replica: ef64bb1caba642ec8cf0f246e313a71d, State: Running, Role: LEADER
I20260807 10:55:39.241014 1910 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef64bb1caba642ec8cf0f246e313a71d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef64bb1caba642ec8cf0f246e313a71d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40019 } }
I20260807 10:55:39.241575 1898 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef64bb1caba642ec8cf0f246e313a71d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:55:39.244444 1913 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef64bb1caba642ec8cf0f246e313a71d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef64bb1caba642ec8cf0f246e313a71d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef64bb1caba642ec8cf0f246e313a71d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40019 } } }
I20260807 10:55:39.245072 1913 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef64bb1caba642ec8cf0f246e313a71d [sys.catalog]: This master's current role is: LEADER
I20260807 10:55:39.244666 1914 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef64bb1caba642ec8cf0f246e313a71d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef64bb1caba642ec8cf0f246e313a71d. Latest consensus state: current_term: 1 leader_uuid: "ef64bb1caba642ec8cf0f246e313a71d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef64bb1caba642ec8cf0f246e313a71d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40019 } } }
I20260807 10:55:39.245538 1914 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef64bb1caba642ec8cf0f246e313a71d [sys.catalog]: This master's current role is: LEADER
I20260807 10:55:39.246286 1926 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:55:39.252032 1926 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:55:39.263487 1926 catalog_manager.cc:1347] Generated new cluster ID: 260efad0f0ce4a3689df66be46785307
I20260807 10:55:39.263631 1926 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:55:39.280020 1926 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:55:39.280781 1926 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:55:39.290822 1926 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef64bb1caba642ec8cf0f246e313a71d: Generated new TSK 0
I20260807 10:55:39.292079 1926 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 10:55:39.309129 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 1792
W20260807 10:55:39.310571 1892 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:43703
W20260807 10:55:39.311414 3367 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 1792: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 10:55:39.311587 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 1792
2026-08-07T10:55:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2467 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:55:39 dist-test-slave-x5fp krb5kdc[1944](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:55:39 dist-test-slave-x5fp krb5kdc[1944](info): set up 2 sockets
Aug 07 10:55:39 dist-test-slave-x5fp krb5kdc[1944](info): commencing operation
krb5kdc: starting...
W20260807 10:55:40.730965 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.373s user 0.000s sys 0.062s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:55:40 dist-test-slave-x5fp krb5kdc[1944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:55:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:55:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 10:55:41.129698 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46145
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46145
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:55:41.519088 1966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:55:41.519455 1966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:55:41.519649 1966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:55:41.527369 1966 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:55:41.527554 1966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:55:41.527657 1966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:55:41.527798 1966 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:55:41.527868 1966 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:55:41.538506 1966 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46145
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:55:41.540113 1966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:55:41.541692 1966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:55:41.555534 1972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:41.555642 1977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:41.556550 1980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:41.557112 1966 server_base.cc:1034] running on GCE node
I20260807 10:55:41.558066 1966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:55:41.559720 1966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:55:41.560989 1966 hybrid_clock.cc:648] HybridClock initialized: now 1786100141560929 us; error 45 us; skew 500 ppm
I20260807 10:55:41.561391 1966 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:55:41 dist-test-slave-x5fp krb5kdc[1944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100141, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:55:41.565078 1966 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 10:55:41.566864 1966 webserver.cc:460] Webserver started at http://127.0.0.1:33983/ using document root <none> and password file <none>
I20260807 10:55:41.567655 1966 fs_manager.cc:359] Metadata directory not provided
I20260807 10:55:41.567780 1966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:55:41.568069 1966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:55:41.571620 1966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "66ec34955c0a47db94b400f510edf22c"
format_stamp: "Formatted at 2026-08-07 10:55:41 on dist-test-slave-x5fp"
server_key: "02cdaa8adc7943ef29454ca3cc25bfa5"
server_key_iv: "8bf74b1d1cbf66a4944cab07b31965e7"
server_key_version: "encryptionkey@0"
I20260807 10:55:41.572465 1966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "66ec34955c0a47db94b400f510edf22c"
format_stamp: "Formatted at 2026-08-07 10:55:41 on dist-test-slave-x5fp"
server_key: "02cdaa8adc7943ef29454ca3cc25bfa5"
server_key_iv: "8bf74b1d1cbf66a4944cab07b31965e7"
server_key_version: "encryptionkey@0"
I20260807 10:55:41.578348 1966 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 10:55:41.582283 1988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:41.583907 1966 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 10:55:41.584088 1966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "66ec34955c0a47db94b400f510edf22c"
format_stamp: "Formatted at 2026-08-07 10:55:41 on dist-test-slave-x5fp"
server_key: "02cdaa8adc7943ef29454ca3cc25bfa5"
server_key_iv: "8bf74b1d1cbf66a4944cab07b31965e7"
server_key_version: "encryptionkey@0"
I20260807 10:55:41.584293 1966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:55:41.623183 1966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:55:41.628644 1966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:55:41.629027 1966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:55:41.650076 1966 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46145
I20260807 10:55:41.650115 2062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46145 every 8 connection(s)
I20260807 10:55:41.651623 1966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:55:41.653075 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 1966
I20260807 10:55:41.653270 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:55:41.653919 3367 external_mini_cluster.cc:1428] Setting key 28e780a0f65369c5036f6689e60f958f
I20260807 10:55:41.656764 2066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 10:55:41 dist-test-slave-x5fp krb5kdc[1944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 10:55:41.672447 2066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66ec34955c0a47db94b400f510edf22c: Bootstrap starting.
I20260807 10:55:41.676122 2066 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66ec34955c0a47db94b400f510edf22c: Neither blocks nor log segments found. Creating new log.
I20260807 10:55:41.677671 2066 log.cc:824] T 00000000000000000000000000000000 P 66ec34955c0a47db94b400f510edf22c: Log is configured to *not* fsync() on all Append() calls
I20260807 10:55:41.681594 2066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66ec34955c0a47db94b400f510edf22c: No bootstrap required, opened a new log
I20260807 10:55:41.681614 2071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:41.655879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36223 (local address 127.0.0.1:46145)
0807 10:55:41.656926 (+ 1047us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:41.656940 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:41.656986 (+ 46us) server_negotiation.cc:407] Connection header received
0807 10:55:41.658674 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:41.658705 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:41.659109 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:41.659460 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:41.661863 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:41.663551 (+ 1688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:41.665630 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:41.666379 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:41.666567 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:41.672477 (+ 5910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:41.672488 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:41.672506 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:55:41.672596 (+ 90us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:41.676095 (+ 3499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:41.677312 (+ 1217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:41.677324 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:41.677332 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:41.677547 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:41.678082 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:41.678086 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:41.678088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:41.679111 (+ 1023us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:41.679465 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:41.679969 (+ 504us) server_negotiation.cc:299] Negotiation successful
0807 10:55:41.680419 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":719,"thread_start_us":477,"threads_started":1}
I20260807 10:55:41.686107 3367 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 10:55:41.692530 2066 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66ec34955c0a47db94b400f510edf22c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66ec34955c0a47db94b400f510edf22c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46145 } }
I20260807 10:55:41.693113 2066 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66ec34955c0a47db94b400f510edf22c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:55:41.693254 2066 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66ec34955c0a47db94b400f510edf22c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66ec34955c0a47db94b400f510edf22c, State: Initialized, Role: FOLLOWER
I20260807 10:55:41.693962 2066 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66ec34955c0a47db94b400f510edf22c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66ec34955c0a47db94b400f510edf22c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46145 } }
I20260807 10:55:41.694223 2066 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66ec34955c0a47db94b400f510edf22c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:55:41.694398 2066 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66ec34955c0a47db94b400f510edf22c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:55:41.694586 2066 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66ec34955c0a47db94b400f510edf22c [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:55:41.697707 2066 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66ec34955c0a47db94b400f510edf22c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66ec34955c0a47db94b400f510edf22c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46145 } }
I20260807 10:55:41.698423 2066 leader_election.cc:302] T 00000000000000000000000000000000 P 66ec34955c0a47db94b400f510edf22c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66ec34955c0a47db94b400f510edf22c; no voters:
I20260807 10:55:41.699220 2066 leader_election.cc:288] T 00000000000000000000000000000000 P 66ec34955c0a47db94b400f510edf22c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:55:41.699383 2084 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66ec34955c0a47db94b400f510edf22c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:55:41.701849 2066 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66ec34955c0a47db94b400f510edf22c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:55:41.703193 2084 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66ec34955c0a47db94b400f510edf22c [term 1 LEADER]: Becoming Leader. State: Replica: 66ec34955c0a47db94b400f510edf22c, State: Running, Role: LEADER
I20260807 10:55:41.703977 2084 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66ec34955c0a47db94b400f510edf22c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66ec34955c0a47db94b400f510edf22c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46145 } }
I20260807 10:55:41.707979 2086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66ec34955c0a47db94b400f510edf22c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66ec34955c0a47db94b400f510edf22c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66ec34955c0a47db94b400f510edf22c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46145 } } }
I20260807 10:55:41.707998 2088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66ec34955c0a47db94b400f510edf22c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66ec34955c0a47db94b400f510edf22c. Latest consensus state: current_term: 1 leader_uuid: "66ec34955c0a47db94b400f510edf22c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66ec34955c0a47db94b400f510edf22c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46145 } } }
I20260807 10:55:41.708498 2086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66ec34955c0a47db94b400f510edf22c [sys.catalog]: This master's current role is: LEADER
I20260807 10:55:41.708498 2088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66ec34955c0a47db94b400f510edf22c [sys.catalog]: This master's current role is: LEADER
I20260807 10:55:41.715566 2071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:41.692196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49025 (local address 127.0.0.1:46145)
0807 10:55:41.692435 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:41.692442 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:41.692955 (+ 513us) server_negotiation.cc:407] Connection header received
0807 10:55:41.693228 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:41.693234 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:41.693356 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:41.693477 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:41.695354 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:41.696545 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:41.697947 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:41.698657 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:41.698799 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:41.703048 (+ 4249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:41.703059 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:41.703067 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:55:41.703135 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:41.708801 (+ 5666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:41.710227 (+ 1426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:41.710237 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:41.710243 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:41.710555 (+ 312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:41.711197 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:41.711206 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:41.711212 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:41.711771 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:41.712328 (+ 557us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:41.714335 (+ 2007us) server_negotiation.cc:299] Negotiation successful
0807 10:55:41.714755 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 10:55:41.722741 2100 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:55:41.729678 2100 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:55:41.739930 2100 catalog_manager.cc:1347] Generated new cluster ID: 1ee6e377b1424aa1a14b32e8d6f372df
I20260807 10:55:41.740087 2100 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:55:41.769996 2100 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:55:41.771611 2100 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:55:41.777696 2100 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66ec34955c0a47db94b400f510edf22c: Generated new TSK 0
I20260807 10:55:41.778636 2100 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 10:55:41.790956 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 1966
I20260807 10:55:41.792318 2058 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:55:41.890040 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 1966
2026-08-07T10:55:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2582 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:55:42 dist-test-slave-x5fp krb5kdc[2122](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:55:42 dist-test-slave-x5fp krb5kdc[2122](info): set up 2 sockets
Aug 07 10:55:42 dist-test-slave-x5fp krb5kdc[2122](info): commencing operation
krb5kdc: starting...
W20260807 10:55:43.279862 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.340s user 0.001s sys 0.061s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:55:43 dist-test-slave-x5fp krb5kdc[2122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:55:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:55:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 10:55:43.681145 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35653
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35653
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:55:44.081846 2149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:55:44.082261 2149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:55:44.082446 2149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:55:44.090272 2149 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:55:44.090487 2149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:55:44.090617 2149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:55:44.090734 2149 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:55:44.090821 2149 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:55:44.102006 2149 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:55:44.103919 2149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:55:44.105448 2149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:55:44.118682 2159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:44.118690 2162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:44.119508 2149 server_base.cc:1034] running on GCE node
W20260807 10:55:44.121001 2160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:44.122259 2149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:55:44.124357 2149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:55:44.125613 2149 hybrid_clock.cc:648] HybridClock initialized: now 1786100144125568 us; error 41 us; skew 500 ppm
I20260807 10:55:44.126030 2149 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:55:44 dist-test-slave-x5fp krb5kdc[2122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100144, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:55:44.129637 2149 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 10:55:44.131407 2149 webserver.cc:460] Webserver started at http://127.0.0.1:40313/ using document root <none> and password file <none>
I20260807 10:55:44.132264 2149 fs_manager.cc:359] Metadata directory not provided
I20260807 10:55:44.132436 2149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:55:44.132869 2149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:55:44.136210 2149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "a266ebb6733b43b7a4c309cff0222248"
format_stamp: "Formatted at 2026-08-07 10:55:44 on dist-test-slave-x5fp"
server_key: "61fb82207543c079943c2bb8b5eeefb4"
server_key_iv: "c2868d8ecb0ce023739a958ab53527c3"
server_key_version: "encryptionkey@0"
I20260807 10:55:44.137136 2149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "a266ebb6733b43b7a4c309cff0222248"
format_stamp: "Formatted at 2026-08-07 10:55:44 on dist-test-slave-x5fp"
server_key: "61fb82207543c079943c2bb8b5eeefb4"
server_key_iv: "c2868d8ecb0ce023739a958ab53527c3"
server_key_version: "encryptionkey@0"
I20260807 10:55:44.143292 2149 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 10:55:44.148276 2173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:44.150012 2149 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 10:55:44.150250 2149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "a266ebb6733b43b7a4c309cff0222248"
format_stamp: "Formatted at 2026-08-07 10:55:44 on dist-test-slave-x5fp"
server_key: "61fb82207543c079943c2bb8b5eeefb4"
server_key_iv: "c2868d8ecb0ce023739a958ab53527c3"
server_key_version: "encryptionkey@0"
I20260807 10:55:44.150485 2149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:55:44.179877 2149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:55:44.185493 2149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:55:44.185822 2149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:55:44.207059 2149 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35653
I20260807 10:55:44.207094 2257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35653 every 8 connection(s)
I20260807 10:55:44.208580 2149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:55:44.212806 2259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:55:44.213933 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 2149
I20260807 10:55:44.214136 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:55:44.214743 3367 external_mini_cluster.cc:1428] Setting key 4bd1a80a5f69ea53be1601929fc4c59e
I20260807 10:55:44.229163 2259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a266ebb6733b43b7a4c309cff0222248: Bootstrap starting.
I20260807 10:55:44.230888 2263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:44.217047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54511 (local address 127.0.0.1:35653)
0807 10:55:44.217989 (+ 942us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:44.218001 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:44.218047 (+ 46us) server_negotiation.cc:407] Connection header received
0807 10:55:44.219811 (+ 1764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:44.219840 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:44.220333 (+ 493us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:44.220781 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:44.221459 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:44.222803 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:44.224878 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:44.225906 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:44.226089 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:44.228031 (+ 1942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:44.228040 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:44.228058 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:44.228800 (+ 742us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:44.228909 (+ 109us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:44.228938 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:44.229043 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:44.229466 (+ 423us) server_negotiation.cc:299] Negotiation successful
0807 10:55:44.229816 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":699,"thread_start_us":397,"threads_started":1}
I20260807 10:55:44.233420 2259 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a266ebb6733b43b7a4c309cff0222248: Neither blocks nor log segments found. Creating new log.
I20260807 10:55:44.234911 2259 log.cc:824] T 00000000000000000000000000000000 P a266ebb6733b43b7a4c309cff0222248: Log is configured to *not* fsync() on all Append() calls
I20260807 10:55:44.235334 3367 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 10:55:44.238770 2259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a266ebb6733b43b7a4c309cff0222248: No bootstrap required, opened a new log
I20260807 10:55:44.247399 2263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:44.240695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36689 (local address 127.0.0.1:35653)
0807 10:55:44.240930 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:44.240939 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:44.241388 (+ 449us) server_negotiation.cc:407] Connection header received
0807 10:55:44.241647 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:44.241651 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:44.241719 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:44.241845 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:44.242414 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:44.243194 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:44.244567 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:44.245149 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:44.245331 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:44.246581 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:44.246586 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:44.246589 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:44.246614 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:44.246661 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:44.246677 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:44.246787 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:44.246951 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 10:55:44.247055 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 10:55:44.247845 2259 raft_consensus.cc:348] T 00000000000000000000000000000000 P a266ebb6733b43b7a4c309cff0222248 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a266ebb6733b43b7a4c309cff0222248" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35653 } }
I20260807 10:55:44.248322 2259 raft_consensus.cc:374] T 00000000000000000000000000000000 P a266ebb6733b43b7a4c309cff0222248 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:55:44.248461 2259 raft_consensus.cc:729] T 00000000000000000000000000000000 P a266ebb6733b43b7a4c309cff0222248 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a266ebb6733b43b7a4c309cff0222248, State: Initialized, Role: FOLLOWER
I20260807 10:55:44.249181 2259 consensus_queue.cc:260] T 00000000000000000000000000000000 P a266ebb6733b43b7a4c309cff0222248 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a266ebb6733b43b7a4c309cff0222248" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35653 } }
I20260807 10:55:44.249506 2259 raft_consensus.cc:388] T 00000000000000000000000000000000 P a266ebb6733b43b7a4c309cff0222248 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:55:44.249670 2259 raft_consensus.cc:482] T 00000000000000000000000000000000 P a266ebb6733b43b7a4c309cff0222248 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:55:44.249831 2259 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a266ebb6733b43b7a4c309cff0222248 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:55:44.252208 2259 raft_consensus.cc:504] T 00000000000000000000000000000000 P a266ebb6733b43b7a4c309cff0222248 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a266ebb6733b43b7a4c309cff0222248" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35653 } }
I20260807 10:55:44.252837 2259 leader_election.cc:302] T 00000000000000000000000000000000 P a266ebb6733b43b7a4c309cff0222248 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a266ebb6733b43b7a4c309cff0222248; no voters:
I20260807 10:55:44.253640 2259 leader_election.cc:288] T 00000000000000000000000000000000 P a266ebb6733b43b7a4c309cff0222248 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:55:44.253810 2271 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a266ebb6733b43b7a4c309cff0222248 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:55:44.255057 2271 raft_consensus.cc:686] T 00000000000000000000000000000000 P a266ebb6733b43b7a4c309cff0222248 [term 1 LEADER]: Becoming Leader. State: Replica: a266ebb6733b43b7a4c309cff0222248, State: Running, Role: LEADER
I20260807 10:55:44.255749 2271 consensus_queue.cc:237] T 00000000000000000000000000000000 P a266ebb6733b43b7a4c309cff0222248 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a266ebb6733b43b7a4c309cff0222248" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35653 } }
I20260807 10:55:44.256500 2259 sys_catalog.cc:564] T 00000000000000000000000000000000 P a266ebb6733b43b7a4c309cff0222248 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:55:44.264191 2272 sys_catalog.cc:455] T 00000000000000000000000000000000 P a266ebb6733b43b7a4c309cff0222248 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a266ebb6733b43b7a4c309cff0222248" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a266ebb6733b43b7a4c309cff0222248" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35653 } } }
I20260807 10:55:44.265082 2272 sys_catalog.cc:458] T 00000000000000000000000000000000 P a266ebb6733b43b7a4c309cff0222248 [sys.catalog]: This master's current role is: LEADER
I20260807 10:55:44.264212 2273 sys_catalog.cc:455] T 00000000000000000000000000000000 P a266ebb6733b43b7a4c309cff0222248 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a266ebb6733b43b7a4c309cff0222248. Latest consensus state: current_term: 1 leader_uuid: "a266ebb6733b43b7a4c309cff0222248" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a266ebb6733b43b7a4c309cff0222248" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35653 } } }
I20260807 10:55:44.266489 2273 sys_catalog.cc:458] T 00000000000000000000000000000000 P a266ebb6733b43b7a4c309cff0222248 [sys.catalog]: This master's current role is: LEADER
W20260807 10:55:44.270601 2287 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a266ebb6733b43b7a4c309cff0222248: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 10:55:44.270772 2287 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 10:55:44.270920 2288 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:55:44.273005 2288 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:55:44.282498 2288 catalog_manager.cc:1347] Generated new cluster ID: e70ab9e440bb4d31a72fff10077d43b0
I20260807 10:55:44.282649 2288 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:55:44.306263 2288 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:55:44.307997 2288 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:55:44.317997 2288 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a266ebb6733b43b7a4c309cff0222248: Generated new TSK 0
I20260807 10:55:44.319232 2288 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 10:55:44.338593 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 2149
W20260807 10:55:44.339928 2250 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:54511
W20260807 10:55:44.341009 3367 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 2149: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 10:55:44.341216 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 2149
2026-08-07T10:55:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2451 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:55:44 dist-test-slave-x5fp krb5kdc[2298](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:55:44 dist-test-slave-x5fp krb5kdc[2298](info): set up 2 sockets
Aug 07 10:55:44 dist-test-slave-x5fp krb5kdc[2298](info): commencing operation
krb5kdc: starting...
W20260807 10:55:45.762753 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.371s user 0.000s sys 0.067s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:55:46 dist-test-slave-x5fp krb5kdc[2298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:55:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:55:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 10:55:46.157771 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39167
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:55:46.552454 2317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:55:46.552953 2317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:55:46.553133 2317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:55:46.561272 2317 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:55:46.561445 2317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:55:46.561539 2317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:55:46.561653 2317 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:55:46.561749 2317 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:55:46.572690 2317 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:55:46.574355 2317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:55:46.575870 2317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:55:46.589962 2327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:46.589882 2330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:46.589938 2328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:46.590868 2317 server_base.cc:1034] running on GCE node
I20260807 10:55:46.591744 2317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:55:46.593474 2317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:55:46.594708 2317 hybrid_clock.cc:648] HybridClock initialized: now 1786100146594653 us; error 44 us; skew 500 ppm
I20260807 10:55:46.595129 2317 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:55:46 dist-test-slave-x5fp krb5kdc[2298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100146, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:55:46.599059 2317 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 10:55:46.601011 2317 webserver.cc:460] Webserver started at http://127.0.0.1:37229/ using document root <none> and password file <none>
I20260807 10:55:46.601869 2317 fs_manager.cc:359] Metadata directory not provided
I20260807 10:55:46.602046 2317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:55:46.602476 2317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:55:46.605993 2317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "1ec2346032d548f994333498852834e3"
format_stamp: "Formatted at 2026-08-07 10:55:46 on dist-test-slave-x5fp"
server_key: "9af96868674e7c0f47be6fbdf1853986"
server_key_iv: "c7e13df1dd6f8e46abb04c4c6a9e0823"
server_key_version: "encryptionkey@0"
I20260807 10:55:46.606935 2317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "1ec2346032d548f994333498852834e3"
format_stamp: "Formatted at 2026-08-07 10:55:46 on dist-test-slave-x5fp"
server_key: "9af96868674e7c0f47be6fbdf1853986"
server_key_iv: "c7e13df1dd6f8e46abb04c4c6a9e0823"
server_key_version: "encryptionkey@0"
I20260807 10:55:46.613049 2317 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 10:55:46.617296 2340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:46.619011 2317 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260807 10:55:46.619228 2317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "1ec2346032d548f994333498852834e3"
format_stamp: "Formatted at 2026-08-07 10:55:46 on dist-test-slave-x5fp"
server_key: "9af96868674e7c0f47be6fbdf1853986"
server_key_iv: "c7e13df1dd6f8e46abb04c4c6a9e0823"
server_key_version: "encryptionkey@0"
I20260807 10:55:46.619488 2317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:55:46.626251 2317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:55:46.631577 2317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:55:46.631959 2317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:55:46.652829 2317 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39167
I20260807 10:55:46.652938 2412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39167 every 8 connection(s)
I20260807 10:55:46.654667 2317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:55:46.659397 2415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:55:46.659528 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 2317
I20260807 10:55:46.659714 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:55:46.660338 3367 external_mini_cluster.cc:1428] Setting key b0d342424d6456256d944597dbaf13ac
I20260807 10:55:46.669121 2418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:46.662451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46235 (local address 127.0.0.1:39167)
0807 10:55:46.663485 (+ 1034us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:46.663501 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:46.663547 (+ 46us) server_negotiation.cc:407] Connection header received
0807 10:55:46.665228 (+ 1681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:46.665254 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:46.665636 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:46.666011 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:46.667260 (+ 1249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:46.667267 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:46.667280 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:46.667298 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:46.667365 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:46.667391 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:46.667585 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:46.667753 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 10:55:46.668002 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":789,"thread_start_us":522,"threads_started":1}
I20260807 10:55:46.673323 3367 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 10:55:46.675684 2415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ec2346032d548f994333498852834e3: Bootstrap starting.
I20260807 10:55:46.680645 2415 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ec2346032d548f994333498852834e3: Neither blocks nor log segments found. Creating new log.
I20260807 10:55:46.682346 2415 log.cc:824] T 00000000000000000000000000000000 P 1ec2346032d548f994333498852834e3: Log is configured to *not* fsync() on all Append() calls
I20260807 10:55:46.682730 2418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:46.679476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44097 (local address 127.0.0.1:39167)
0807 10:55:46.679707 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:46.679715 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:46.680159 (+ 444us) server_negotiation.cc:407] Connection header received
0807 10:55:46.680322 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:46.680332 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:46.680426 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:46.680579 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:46.681895 (+ 1316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:46.681902 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:46.681907 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:46.681927 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:46.681987 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:46.682017 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:46.682175 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:46.682341 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 10:55:46.682424 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 10:55:46.685997 2415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ec2346032d548f994333498852834e3: No bootstrap required, opened a new log
I20260807 10:55:46.694514 2415 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ec2346032d548f994333498852834e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ec2346032d548f994333498852834e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39167 } }
I20260807 10:55:46.695022 2415 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ec2346032d548f994333498852834e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:55:46.695165 2415 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ec2346032d548f994333498852834e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ec2346032d548f994333498852834e3, State: Initialized, Role: FOLLOWER
I20260807 10:55:46.696019 2415 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ec2346032d548f994333498852834e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ec2346032d548f994333498852834e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39167 } }
I20260807 10:55:46.696374 2415 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ec2346032d548f994333498852834e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:55:46.696579 2415 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ec2346032d548f994333498852834e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:55:46.696798 2415 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ec2346032d548f994333498852834e3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:55:46.699383 2415 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ec2346032d548f994333498852834e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ec2346032d548f994333498852834e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39167 } }
I20260807 10:55:46.700034 2415 leader_election.cc:302] T 00000000000000000000000000000000 P 1ec2346032d548f994333498852834e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ec2346032d548f994333498852834e3; no voters:
I20260807 10:55:46.700821 2415 leader_election.cc:288] T 00000000000000000000000000000000 P 1ec2346032d548f994333498852834e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:55:46.701026 2427 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ec2346032d548f994333498852834e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:55:46.701864 2427 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ec2346032d548f994333498852834e3 [term 1 LEADER]: Becoming Leader. State: Replica: 1ec2346032d548f994333498852834e3, State: Running, Role: LEADER
I20260807 10:55:46.702554 2427 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ec2346032d548f994333498852834e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ec2346032d548f994333498852834e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39167 } }
I20260807 10:55:46.703660 2415 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ec2346032d548f994333498852834e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:55:46.706202 2430 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ec2346032d548f994333498852834e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ec2346032d548f994333498852834e3. Latest consensus state: current_term: 1 leader_uuid: "1ec2346032d548f994333498852834e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ec2346032d548f994333498852834e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39167 } } }
I20260807 10:55:46.706192 2428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ec2346032d548f994333498852834e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ec2346032d548f994333498852834e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ec2346032d548f994333498852834e3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39167 } } }
I20260807 10:55:46.707512 2428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ec2346032d548f994333498852834e3 [sys.catalog]: This master's current role is: LEADER
I20260807 10:55:46.707507 2430 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ec2346032d548f994333498852834e3 [sys.catalog]: This master's current role is: LEADER
I20260807 10:55:46.709510 2437 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:55:46.715833 2437 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:55:46.727650 2437 catalog_manager.cc:1347] Generated new cluster ID: 3e095d7559c2489cb05f43efea9eaffc
I20260807 10:55:46.727826 2437 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:55:46.743963 2437 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:55:46.744655 2437 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:55:46.753041 2437 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ec2346032d548f994333498852834e3: Generated new TSK 0
I20260807 10:55:46.754052 2437 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 10:55:46.772080 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 2317
W20260807 10:55:46.773375 2407 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46235
W20260807 10:55:46.774209 3367 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 2317: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 10:55:46.774412 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 2317
2026-08-07T10:55:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2430 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:55:46 dist-test-slave-x5fp krb5kdc[2456](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:55:46 dist-test-slave-x5fp krb5kdc[2456](info): set up 2 sockets
Aug 07 10:55:46 dist-test-slave-x5fp krb5kdc[2456](info): commencing operation
krb5kdc: starting...
W20260807 10:55:48.221858 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.399s user 0.006s sys 0.061s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:55:48 dist-test-slave-x5fp krb5kdc[2456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:55:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:55:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 10:55:48.625144 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41175
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41175
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:55:49.001993 2475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:55:49.002401 2475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:55:49.002589 2475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:55:49.010695 2475 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:55:49.010895 2475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:55:49.011015 2475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:55:49.011128 2475 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:55:49.011209 2475 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:55:49.021899 2475 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41175
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:55:49.023452 2475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:55:49.024971 2475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:55:49.039016 2482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:49.038985 2486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:49.039817 2483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:49.040201 2475 server_base.cc:1034] running on GCE node
I20260807 10:55:49.041455 2475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:55:49.043423 2475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:55:49.044698 2475 hybrid_clock.cc:648] HybridClock initialized: now 1786100149044633 us; error 58 us; skew 500 ppm
I20260807 10:55:49.045215 2475 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:55:49 dist-test-slave-x5fp krb5kdc[2456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100149, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:55:49.048676 2475 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 10:55:49.050449 2475 webserver.cc:460] Webserver started at http://127.0.0.1:38289/ using document root <none> and password file <none>
I20260807 10:55:49.051188 2475 fs_manager.cc:359] Metadata directory not provided
I20260807 10:55:49.051306 2475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:55:49.051595 2475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:55:49.055097 2475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "99f32b2b52e84f43a2037e6b826be3e2"
format_stamp: "Formatted at 2026-08-07 10:55:49 on dist-test-slave-x5fp"
server_key: "b068b131d79c79be06ff3494dd4ad697"
server_key_iv: "a6f19b2c2e197e191703c799da25de14"
server_key_version: "encryptionkey@0"
I20260807 10:55:49.055925 2475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "99f32b2b52e84f43a2037e6b826be3e2"
format_stamp: "Formatted at 2026-08-07 10:55:49 on dist-test-slave-x5fp"
server_key: "b068b131d79c79be06ff3494dd4ad697"
server_key_iv: "a6f19b2c2e197e191703c799da25de14"
server_key_version: "encryptionkey@0"
I20260807 10:55:49.062034 2475 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 10:55:49.066429 2500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:49.068166 2475 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 10:55:49.068416 2475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "99f32b2b52e84f43a2037e6b826be3e2"
format_stamp: "Formatted at 2026-08-07 10:55:49 on dist-test-slave-x5fp"
server_key: "b068b131d79c79be06ff3494dd4ad697"
server_key_iv: "a6f19b2c2e197e191703c799da25de14"
server_key_version: "encryptionkey@0"
I20260807 10:55:49.068640 2475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:55:49.084353 2475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:55:49.089440 2475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:55:49.089789 2475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:55:49.113247 2475 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41175
I20260807 10:55:49.113283 2569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41175 every 8 connection(s)
I20260807 10:55:49.115075 2475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:55:49.119549 2571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:55:49.124900 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 2475
I20260807 10:55:49.125139 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:55:49.125898 3367 external_mini_cluster.cc:1428] Setting key 9a429b1bfdb653942cd51ebef760fcbd
I20260807 10:55:49.132241 2571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99f32b2b52e84f43a2037e6b826be3e2: Bootstrap starting.
I20260807 10:55:49.136426 2571 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99f32b2b52e84f43a2037e6b826be3e2: Neither blocks nor log segments found. Creating new log.
I20260807 10:55:49.137899 2571 log.cc:824] T 00000000000000000000000000000000 P 99f32b2b52e84f43a2037e6b826be3e2: Log is configured to *not* fsync() on all Append() calls
I20260807 10:55:49.142278 2571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99f32b2b52e84f43a2037e6b826be3e2: No bootstrap required, opened a new log
Aug 07 10:55:49 dist-test-slave-x5fp krb5kdc[2456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 10:55:49.151847 2571 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99f32b2b52e84f43a2037e6b826be3e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99f32b2b52e84f43a2037e6b826be3e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41175 } }
I20260807 10:55:49.152365 2571 raft_consensus.cc:374] T 00000000000000000000000000000000 P 99f32b2b52e84f43a2037e6b826be3e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:55:49.152559 2571 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99f32b2b52e84f43a2037e6b826be3e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99f32b2b52e84f43a2037e6b826be3e2, State: Initialized, Role: FOLLOWER
I20260807 10:55:49.153409 2571 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99f32b2b52e84f43a2037e6b826be3e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99f32b2b52e84f43a2037e6b826be3e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41175 } }
I20260807 10:55:49.153767 2571 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99f32b2b52e84f43a2037e6b826be3e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:55:49.153967 2571 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99f32b2b52e84f43a2037e6b826be3e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:55:49.154206 2571 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99f32b2b52e84f43a2037e6b826be3e2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:55:49.154387 2574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:49.128358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.203:39031 (local address 127.0.0.1:41175)
0807 10:55:49.129476 (+ 1118us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:49.129490 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:49.129547 (+ 57us) server_negotiation.cc:407] Connection header received
0807 10:55:49.131537 (+ 1990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:49.131564 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:49.132025 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:49.132513 (+ 488us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:49.134561 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:49.135962 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:49.137985 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:49.139138 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:49.139398 (+ 260us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:49.145358 (+ 5960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:49.145367 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:49.145384 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:55:49.145442 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:49.149149 (+ 3707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:49.150163 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:49.150173 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:49.150182 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:49.150381 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:49.150913 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:49.150917 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:49.150919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:49.151524 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:49.151910 (+ 386us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:49.152451 (+ 541us) server_negotiation.cc:299] Negotiation successful
0807 10:55:49.152906 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":832,"thread_start_us":548,"threads_started":1}
I20260807 10:55:49.157158 2571 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99f32b2b52e84f43a2037e6b826be3e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99f32b2b52e84f43a2037e6b826be3e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41175 } }
I20260807 10:55:49.157903 2571 leader_election.cc:302] T 00000000000000000000000000000000 P 99f32b2b52e84f43a2037e6b826be3e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99f32b2b52e84f43a2037e6b826be3e2; no voters:
I20260807 10:55:49.158722 2571 leader_election.cc:288] T 00000000000000000000000000000000 P 99f32b2b52e84f43a2037e6b826be3e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:55:49.158866 3367 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 10:55:49.158856 2576 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99f32b2b52e84f43a2037e6b826be3e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:55:49.161877 2571 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99f32b2b52e84f43a2037e6b826be3e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:55:49.163930 2576 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99f32b2b52e84f43a2037e6b826be3e2 [term 1 LEADER]: Becoming Leader. State: Replica: 99f32b2b52e84f43a2037e6b826be3e2, State: Running, Role: LEADER
I20260807 10:55:49.164991 2576 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99f32b2b52e84f43a2037e6b826be3e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99f32b2b52e84f43a2037e6b826be3e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41175 } }
I20260807 10:55:49.178611 2580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99f32b2b52e84f43a2037e6b826be3e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99f32b2b52e84f43a2037e6b826be3e2. Latest consensus state: current_term: 1 leader_uuid: "99f32b2b52e84f43a2037e6b826be3e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99f32b2b52e84f43a2037e6b826be3e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41175 } } }
I20260807 10:55:49.179203 2580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99f32b2b52e84f43a2037e6b826be3e2 [sys.catalog]: This master's current role is: LEADER
I20260807 10:55:49.183791 2574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:49.165042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.203:52001 (local address 127.0.0.1:41175)
0807 10:55:49.165229 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:49.165234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:49.165249 (+ 15us) server_negotiation.cc:407] Connection header received
0807 10:55:49.165306 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:49.165310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:49.165380 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:49.165502 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:49.167306 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:49.168211 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:49.169625 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:49.170106 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:49.170289 (+ 183us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:49.174243 (+ 3954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:49.174248 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:49.174251 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:55:49.174331 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:49.177915 (+ 3584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:49.179457 (+ 1542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:49.179466 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:49.179472 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:49.179683 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:49.180458 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:49.180467 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:49.180473 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:49.181062 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:49.181401 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:49.182948 (+ 1547us) server_negotiation.cc:299] Negotiation successful
0807 10:55:49.183226 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 10:55:49.185510 2579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99f32b2b52e84f43a2037e6b826be3e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "99f32b2b52e84f43a2037e6b826be3e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99f32b2b52e84f43a2037e6b826be3e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41175 } } }
I20260807 10:55:49.185921 2579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99f32b2b52e84f43a2037e6b826be3e2 [sys.catalog]: This master's current role is: LEADER
I20260807 10:55:49.186990 2602 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:55:49.192667 2602 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:55:49.202963 2602 catalog_manager.cc:1347] Generated new cluster ID: 6ed8420e468d428cbdea29f7f43277d3
I20260807 10:55:49.203104 2602 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:55:49.234877 2602 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:55:49.236318 2602 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:55:49.245667 2602 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 99f32b2b52e84f43a2037e6b826be3e2: Generated new TSK 0
I20260807 10:55:49.246663 2602 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 10:55:49.268679 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 2475
I20260807 10:55:49.270220 2565 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:55:49.368592 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 2475
W20260807 10:55:49.399878 2452 connection.cc:535] client connection to 127.0.0.1:41175 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41175): Connection reset by peer (error 104)
2026-08-07T10:55:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2598 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:55:49 dist-test-slave-x5fp krb5kdc[2613](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:55:49 dist-test-slave-x5fp krb5kdc[2613](info): set up 2 sockets
Aug 07 10:55:49 dist-test-slave-x5fp krb5kdc[2613](info): commencing operation
krb5kdc: starting...
W20260807 10:55:50.795451 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.375s user 0.003s sys 0.063s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:55:51 dist-test-slave-x5fp krb5kdc[2613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:55:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:55:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 10:55:51.196684 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40241
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40241
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:55:51.586027 2630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:55:51.586428 2630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:55:51.586616 2630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:55:51.594460 2630 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:55:51.594656 2630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:55:51.594779 2630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:55:51.594913 2630 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:55:51.594998 2630 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:55:51.605670 2630 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:55:51.607165 2630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:55:51.608629 2630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:55:51.622113 2639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:51.622128 2638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:51.622457 2641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:51.623760 2630 server_base.cc:1034] running on GCE node
I20260807 10:55:51.624974 2630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:55:51.626981 2630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:55:51.628319 2630 hybrid_clock.cc:648] HybridClock initialized: now 1786100151628245 us; error 72 us; skew 500 ppm
I20260807 10:55:51.628834 2630 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:55:51 dist-test-slave-x5fp krb5kdc[2613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100151, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:55:51.632643 2630 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 10:55:51.634542 2630 webserver.cc:460] Webserver started at http://127.0.0.1:32985/ using document root <none> and password file <none>
I20260807 10:55:51.635329 2630 fs_manager.cc:359] Metadata directory not provided
I20260807 10:55:51.635489 2630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:55:51.635829 2630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:55:51.639180 2630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "6e7d46447c794ece97e59eed50587beb"
format_stamp: "Formatted at 2026-08-07 10:55:51 on dist-test-slave-x5fp"
server_key: "7e389fd9c64066bc3057e534eca477c2"
server_key_iv: "ced30b49a1c537bde2fdbc185f053520"
server_key_version: "encryptionkey@0"
I20260807 10:55:51.640043 2630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "6e7d46447c794ece97e59eed50587beb"
format_stamp: "Formatted at 2026-08-07 10:55:51 on dist-test-slave-x5fp"
server_key: "7e389fd9c64066bc3057e534eca477c2"
server_key_iv: "ced30b49a1c537bde2fdbc185f053520"
server_key_version: "encryptionkey@0"
I20260807 10:55:51.645968 2630 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260807 10:55:51.650211 2654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:51.651788 2630 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260807 10:55:51.652020 2630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "6e7d46447c794ece97e59eed50587beb"
format_stamp: "Formatted at 2026-08-07 10:55:51 on dist-test-slave-x5fp"
server_key: "7e389fd9c64066bc3057e534eca477c2"
server_key_iv: "ced30b49a1c537bde2fdbc185f053520"
server_key_version: "encryptionkey@0"
I20260807 10:55:51.652263 2630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:55:51.675323 2630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:55:51.680399 2630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:55:51.680796 2630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:55:51.701645 2630 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40241
I20260807 10:55:51.701665 2731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40241 every 8 connection(s)
I20260807 10:55:51.703162 2630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:55:51.707201 2732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:55:51.707865 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 2630
I20260807 10:55:51.708323 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:55:51.708933 3367 external_mini_cluster.cc:1428] Setting key 5412b5f3ec6a4c961a7dcf1ec68e5de8
I20260807 10:55:51.723160 2732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e7d46447c794ece97e59eed50587beb: Bootstrap starting.
I20260807 10:55:51.725049 2736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:51.710894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.203:33369 (local address 127.0.0.1:40241)
0807 10:55:51.711787 (+ 893us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:51.711799 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:51.711848 (+ 49us) server_negotiation.cc:407] Connection header received
0807 10:55:51.714104 (+ 2256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:51.714135 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:51.714703 (+ 568us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:51.715129 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:51.716002 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:51.717550 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:51.719742 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:51.720657 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:51.720913 (+ 256us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:51.722384 (+ 1471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:51.722395 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:51.722413 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:51.723021 (+ 608us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:51.723108 (+ 87us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:51.723135 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:51.723261 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:51.723649 (+ 388us) server_negotiation.cc:299] Negotiation successful
0807 10:55:51.723912 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":407,"threads_started":1}
I20260807 10:55:51.726991 2732 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e7d46447c794ece97e59eed50587beb: Neither blocks nor log segments found. Creating new log.
I20260807 10:55:51.728313 2732 log.cc:824] T 00000000000000000000000000000000 P 6e7d46447c794ece97e59eed50587beb: Log is configured to *not* fsync() on all Append() calls
I20260807 10:55:51.729482 3367 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 10:55:51.731585 2732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e7d46447c794ece97e59eed50587beb: No bootstrap required, opened a new log
I20260807 10:55:51.741772 2736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:51.734436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.203:44393 (local address 127.0.0.1:40241)
0807 10:55:51.734669 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:51.734676 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:51.734970 (+ 294us) server_negotiation.cc:407] Connection header received
0807 10:55:51.735224 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:51.735228 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:51.735306 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:51.735434 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:51.735995 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:51.736891 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:51.738342 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:51.738949 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:51.739160 (+ 211us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:51.740906 (+ 1746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:51.740914 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:51.740918 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:51.740945 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:51.741004 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:51.741024 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:51.741162 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:51.741397 (+ 235us) server_negotiation.cc:299] Negotiation successful
0807 10:55:51.741469 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 10:55:51.741349 2732 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e7d46447c794ece97e59eed50587beb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e7d46447c794ece97e59eed50587beb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40241 } }
I20260807 10:55:51.742064 2732 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e7d46447c794ece97e59eed50587beb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:55:51.742214 2732 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e7d46447c794ece97e59eed50587beb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e7d46447c794ece97e59eed50587beb, State: Initialized, Role: FOLLOWER
I20260807 10:55:51.743037 2732 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e7d46447c794ece97e59eed50587beb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e7d46447c794ece97e59eed50587beb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40241 } }
I20260807 10:55:51.743357 2732 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e7d46447c794ece97e59eed50587beb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:55:51.743546 2732 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e7d46447c794ece97e59eed50587beb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:55:51.743762 2732 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e7d46447c794ece97e59eed50587beb [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:55:51.746764 2732 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e7d46447c794ece97e59eed50587beb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e7d46447c794ece97e59eed50587beb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40241 } }
I20260807 10:55:51.747429 2732 leader_election.cc:302] T 00000000000000000000000000000000 P 6e7d46447c794ece97e59eed50587beb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e7d46447c794ece97e59eed50587beb; no voters:
I20260807 10:55:51.748324 2732 leader_election.cc:288] T 00000000000000000000000000000000 P 6e7d46447c794ece97e59eed50587beb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:55:51.748548 2745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e7d46447c794ece97e59eed50587beb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:55:51.749950 2745 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e7d46447c794ece97e59eed50587beb [term 1 LEADER]: Becoming Leader. State: Replica: 6e7d46447c794ece97e59eed50587beb, State: Running, Role: LEADER
I20260807 10:55:51.750767 2745 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e7d46447c794ece97e59eed50587beb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e7d46447c794ece97e59eed50587beb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40241 } }
I20260807 10:55:51.751063 2732 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e7d46447c794ece97e59eed50587beb [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:55:51.754778 2747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e7d46447c794ece97e59eed50587beb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e7d46447c794ece97e59eed50587beb. Latest consensus state: current_term: 1 leader_uuid: "6e7d46447c794ece97e59eed50587beb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e7d46447c794ece97e59eed50587beb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40241 } } }
I20260807 10:55:51.755124 2746 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e7d46447c794ece97e59eed50587beb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e7d46447c794ece97e59eed50587beb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e7d46447c794ece97e59eed50587beb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40241 } } }
I20260807 10:55:51.755455 2746 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e7d46447c794ece97e59eed50587beb [sys.catalog]: This master's current role is: LEADER
I20260807 10:55:51.755461 2747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e7d46447c794ece97e59eed50587beb [sys.catalog]: This master's current role is: LEADER
I20260807 10:55:51.762624 2754 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:55:51.768239 2754 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:55:51.778657 2754 catalog_manager.cc:1347] Generated new cluster ID: db21f70d11464fc8b8445b231f852888
I20260807 10:55:51.778813 2754 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:55:51.791425 2754 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:55:51.793303 2754 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:55:51.803344 2754 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e7d46447c794ece97e59eed50587beb: Generated new TSK 0
I20260807 10:55:51.804464 2754 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 10:55:51.823056 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 2630
W20260807 10:55:51.824357 2726 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.203:33369
W20260807 10:55:51.825392 3367 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 2630: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 10:55:51.825615 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 2630
W20260807 10:55:51.854875 2611 connection.cc:535] client connection to 127.0.0.1:40241 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40241): Connection reset by peer (error 104)
2026-08-07T10:55:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2454 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:55:51 dist-test-slave-x5fp krb5kdc[2775](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:55:51 dist-test-slave-x5fp krb5kdc[2775](info): set up 2 sockets
Aug 07 10:55:51 dist-test-slave-x5fp krb5kdc[2775](info): commencing operation
krb5kdc: starting...
W20260807 10:55:53.266577 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.391s user 0.002s sys 0.065s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:55:53 dist-test-slave-x5fp krb5kdc[2775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:55:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:55:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 10:55:53.673337 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34699
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:55:54.055475 2788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:55:54.055804 2788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:55:54.055912 2788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:55:54.063880 2788 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:55:54.064038 2788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:55:54.064090 2788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:55:54.064147 2788 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:55:54.064194 2788 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:55:54.075062 2788 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:55:54.076478 2788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:55:54.077821 2788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:55:54.091820 2795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:54.091845 2798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:54.091890 2796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:54.092854 2788 server_base.cc:1034] running on GCE node
I20260807 10:55:54.093744 2788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:55:54.095315 2788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:55:54.096562 2788 hybrid_clock.cc:648] HybridClock initialized: now 1786100154096518 us; error 32 us; skew 500 ppm
I20260807 10:55:54.097031 2788 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:55:54 dist-test-slave-x5fp krb5kdc[2775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100154, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:55:54.100558 2788 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 10:55:54.102342 2788 webserver.cc:460] Webserver started at http://127.0.0.1:43221/ using document root <none> and password file <none>
I20260807 10:55:54.103169 2788 fs_manager.cc:359] Metadata directory not provided
I20260807 10:55:54.103289 2788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:55:54.103566 2788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:55:54.106793 2788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "33e7716935c642c4a53ab5bde46eb1a9"
format_stamp: "Formatted at 2026-08-07 10:55:54 on dist-test-slave-x5fp"
server_key: "5eb0783019c20aa49f666bff55989472"
server_key_iv: "bc329eb49503a8b849df93a778390f8d"
server_key_version: "encryptionkey@0"
I20260807 10:55:54.107651 2788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "33e7716935c642c4a53ab5bde46eb1a9"
format_stamp: "Formatted at 2026-08-07 10:55:54 on dist-test-slave-x5fp"
server_key: "5eb0783019c20aa49f666bff55989472"
server_key_iv: "bc329eb49503a8b849df93a778390f8d"
server_key_version: "encryptionkey@0"
I20260807 10:55:54.113467 2788 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 10:55:54.117372 2809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:54.118884 2788 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 10:55:54.119092 2788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "33e7716935c642c4a53ab5bde46eb1a9"
format_stamp: "Formatted at 2026-08-07 10:55:54 on dist-test-slave-x5fp"
server_key: "5eb0783019c20aa49f666bff55989472"
server_key_iv: "bc329eb49503a8b849df93a778390f8d"
server_key_version: "encryptionkey@0"
I20260807 10:55:54.119307 2788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:55:54.125927 2788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:55:54.130885 2788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:55:54.131246 2788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:55:54.152509 2788 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34699
I20260807 10:55:54.152544 2891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34699 every 8 connection(s)
I20260807 10:55:54.154230 2788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:55:54.158334 2895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:55:54.164139 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 2788
I20260807 10:55:54.164353 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:55:54.164892 3367 external_mini_cluster.cc:1428] Setting key 749a521a33e8208eb54c41d57fb2be58
I20260807 10:55:54.170192 2895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33e7716935c642c4a53ab5bde46eb1a9: Bootstrap starting.
I20260807 10:55:54.173697 2900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:54.167054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.203:48981 (local address 127.0.0.1:34699)
0807 10:55:54.167983 (+ 929us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:54.167995 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:54.168039 (+ 44us) server_negotiation.cc:407] Connection header received
0807 10:55:54.169842 (+ 1803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:54.169866 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:54.170221 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:54.170621 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:54.171876 (+ 1255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:54.171885 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:54.171902 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:54.171921 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:54.171998 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:54.172022 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:54.172224 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:54.172388 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 10:55:54.172636 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":436,"threads_started":1}
I20260807 10:55:54.174366 2895 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33e7716935c642c4a53ab5bde46eb1a9: Neither blocks nor log segments found. Creating new log.
I20260807 10:55:54.175581 2895 log.cc:824] T 00000000000000000000000000000000 P 33e7716935c642c4a53ab5bde46eb1a9: Log is configured to *not* fsync() on all Append() calls
I20260807 10:55:54.177876 3367 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 10:55:54.179345 2895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33e7716935c642c4a53ab5bde46eb1a9: No bootstrap required, opened a new log
I20260807 10:55:54.186296 2900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:54.183520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.203:45327 (local address 127.0.0.1:34699)
0807 10:55:54.183730 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:54.183737 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:54.184145 (+ 408us) server_negotiation.cc:407] Connection header received
0807 10:55:54.184290 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:54.184295 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:54.184362 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:54.184481 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:54.185524 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:54.185531 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:54.185536 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:54.185555 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:54.185609 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:54.185637 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:54.185773 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:54.185916 (+ 143us) server_negotiation.cc:299] Negotiation successful
0807 10:55:54.185986 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 10:55:54.190026 2895 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33e7716935c642c4a53ab5bde46eb1a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33e7716935c642c4a53ab5bde46eb1a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34699 } }
I20260807 10:55:54.190434 2895 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33e7716935c642c4a53ab5bde46eb1a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:55:54.190513 2895 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33e7716935c642c4a53ab5bde46eb1a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33e7716935c642c4a53ab5bde46eb1a9, State: Initialized, Role: FOLLOWER
I20260807 10:55:54.191066 2895 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33e7716935c642c4a53ab5bde46eb1a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33e7716935c642c4a53ab5bde46eb1a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34699 } }
I20260807 10:55:54.191298 2895 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33e7716935c642c4a53ab5bde46eb1a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:55:54.191413 2895 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33e7716935c642c4a53ab5bde46eb1a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:55:54.191552 2895 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33e7716935c642c4a53ab5bde46eb1a9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:55:54.193965 2895 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33e7716935c642c4a53ab5bde46eb1a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33e7716935c642c4a53ab5bde46eb1a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34699 } }
I20260807 10:55:54.194481 2895 leader_election.cc:302] T 00000000000000000000000000000000 P 33e7716935c642c4a53ab5bde46eb1a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33e7716935c642c4a53ab5bde46eb1a9; no voters:
I20260807 10:55:54.195068 2895 leader_election.cc:288] T 00000000000000000000000000000000 P 33e7716935c642c4a53ab5bde46eb1a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:55:54.195241 2910 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33e7716935c642c4a53ab5bde46eb1a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:55:54.196611 2910 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33e7716935c642c4a53ab5bde46eb1a9 [term 1 LEADER]: Becoming Leader. State: Replica: 33e7716935c642c4a53ab5bde46eb1a9, State: Running, Role: LEADER
I20260807 10:55:54.197301 2910 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33e7716935c642c4a53ab5bde46eb1a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33e7716935c642c4a53ab5bde46eb1a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34699 } }
I20260807 10:55:54.197886 2895 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33e7716935c642c4a53ab5bde46eb1a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:55:54.201077 2911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33e7716935c642c4a53ab5bde46eb1a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "33e7716935c642c4a53ab5bde46eb1a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33e7716935c642c4a53ab5bde46eb1a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34699 } } }
I20260807 10:55:54.201854 2911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33e7716935c642c4a53ab5bde46eb1a9 [sys.catalog]: This master's current role is: LEADER
I20260807 10:55:54.200991 2913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33e7716935c642c4a53ab5bde46eb1a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33e7716935c642c4a53ab5bde46eb1a9. Latest consensus state: current_term: 1 leader_uuid: "33e7716935c642c4a53ab5bde46eb1a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33e7716935c642c4a53ab5bde46eb1a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34699 } } }
I20260807 10:55:54.202157 2913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33e7716935c642c4a53ab5bde46eb1a9 [sys.catalog]: This master's current role is: LEADER
W20260807 10:55:54.211766 2931 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 33e7716935c642c4a53ab5bde46eb1a9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 10:55:54.212106 2931 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 10:55:54.214217 2920 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:55:54.216648 2920 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:55:54.227600 2920 catalog_manager.cc:1347] Generated new cluster ID: b1a14cf0e08b468c8776d57aa6ad81c1
I20260807 10:55:54.227751 2920 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:55:54.251034 2920 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:55:54.251590 2920 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:55:54.259611 2920 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33e7716935c642c4a53ab5bde46eb1a9: Generated new TSK 0
I20260807 10:55:54.260776 2920 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 10:55:54.276376 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 2788
W20260807 10:55:54.277854 2884 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.203:48981
W20260807 10:55:54.278708 3367 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 2788: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 10:55:54.278890 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 2788
2026-08-07T10:55:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2451 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:55:54 dist-test-slave-x5fp krb5kdc[2944](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:55:54 dist-test-slave-x5fp krb5kdc[2944](info): set up 2 sockets
Aug 07 10:55:54 dist-test-slave-x5fp krb5kdc[2944](info): commencing operation
krb5kdc: starting...
W20260807 10:55:55.695290 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.369s user 0.004s sys 0.062s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:55:55 dist-test-slave-x5fp krb5kdc[2944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:55:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:55:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 10:55:56.102710 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33799
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:55:56.493965 2968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:55:56.494309 2968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:55:56.494498 2968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:55:56.502416 2968 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:55:56.502579 2968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:55:56.502694 2968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:55:56.502817 2968 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:55:56.502911 2968 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:55:56.514160 2968 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33799
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:55:56.515774 2968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:55:56.517366 2968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:55:56.531247 2974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:56.531284 2977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:56.531654 2979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:56.532869 2968 server_base.cc:1034] running on GCE node
I20260807 10:55:56.533851 2968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:55:56.535673 2968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:55:56.536928 2968 hybrid_clock.cc:648] HybridClock initialized: now 1786100156536853 us; error 66 us; skew 500 ppm
I20260807 10:55:56.537377 2968 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:55:56 dist-test-slave-x5fp krb5kdc[2944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100156, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:55:56.541065 2968 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 10:55:56.542761 2968 webserver.cc:460] Webserver started at http://127.0.0.1:40293/ using document root <none> and password file <none>
I20260807 10:55:56.543594 2968 fs_manager.cc:359] Metadata directory not provided
I20260807 10:55:56.543764 2968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:55:56.544119 2968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:55:56.547617 2968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "5505dd289b804a26a17a5a45185257e2"
format_stamp: "Formatted at 2026-08-07 10:55:56 on dist-test-slave-x5fp"
server_key: "548d45796780b5749aff98974519632e"
server_key_iv: "a3e875064fdb8faede39bb5315f1f88f"
server_key_version: "encryptionkey@0"
I20260807 10:55:56.548484 2968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "5505dd289b804a26a17a5a45185257e2"
format_stamp: "Formatted at 2026-08-07 10:55:56 on dist-test-slave-x5fp"
server_key: "548d45796780b5749aff98974519632e"
server_key_iv: "a3e875064fdb8faede39bb5315f1f88f"
server_key_version: "encryptionkey@0"
I20260807 10:55:56.554503 2968 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260807 10:55:56.558763 2990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:56.560444 2968 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 10:55:56.560683 2968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "5505dd289b804a26a17a5a45185257e2"
format_stamp: "Formatted at 2026-08-07 10:55:56 on dist-test-slave-x5fp"
server_key: "548d45796780b5749aff98974519632e"
server_key_iv: "a3e875064fdb8faede39bb5315f1f88f"
server_key_version: "encryptionkey@0"
I20260807 10:55:56.560984 2968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:55:56.587078 2968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:55:56.592036 2968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:55:56.592374 2968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:55:56.613514 2968 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33799
I20260807 10:55:56.613575 3065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33799 every 8 connection(s)
I20260807 10:55:56.615154 2968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:55:56.616426 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 2968
I20260807 10:55:56.616688 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:55:56.617380 3367 external_mini_cluster.cc:1428] Setting key 7ea76f534daa9f5eb0d5b2bd6f334904
I20260807 10:55:56.620934 3067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 10:55:56 dist-test-slave-x5fp krb5kdc[2944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 10:55:56.634053 3067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5505dd289b804a26a17a5a45185257e2: Bootstrap starting.
I20260807 10:55:56.638329 3067 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5505dd289b804a26a17a5a45185257e2: Neither blocks nor log segments found. Creating new log.
I20260807 10:55:56.639616 3067 log.cc:824] T 00000000000000000000000000000000 P 5505dd289b804a26a17a5a45185257e2: Log is configured to *not* fsync() on all Append() calls
I20260807 10:55:56.642999 3072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:56.619559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.203:58953 (local address 127.0.0.1:33799)
0807 10:55:56.620415 (+ 856us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:56.620427 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:56.620470 (+ 43us) server_negotiation.cc:407] Connection header received
0807 10:55:56.622388 (+ 1918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:56.622412 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:56.622829 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:56.623224 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:56.624830 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:56.626360 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:56.627731 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:56.628476 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:56.628689 (+ 213us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:56.633772 (+ 5083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:56.633784 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:56.633808 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:55:56.633859 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:56.638046 (+ 4187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:56.639146 (+ 1100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:56.639154 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:56.639161 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:56.639349 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:56.639874 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:56.639880 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:56.639882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:56.640684 (+ 802us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:56.641084 (+ 400us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:56.641565 (+ 481us) server_negotiation.cc:299] Negotiation successful
0807 10:55:56.641916 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":372,"threads_started":1}
I20260807 10:55:56.643255 3067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5505dd289b804a26a17a5a45185257e2: No bootstrap required, opened a new log
I20260807 10:55:56.647511 3367 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 10:55:56.653909 3067 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5505dd289b804a26a17a5a45185257e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5505dd289b804a26a17a5a45185257e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33799 } }
I20260807 10:55:56.654481 3067 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5505dd289b804a26a17a5a45185257e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:55:56.654619 3067 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5505dd289b804a26a17a5a45185257e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5505dd289b804a26a17a5a45185257e2, State: Initialized, Role: FOLLOWER
I20260807 10:55:56.655287 3067 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5505dd289b804a26a17a5a45185257e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5505dd289b804a26a17a5a45185257e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33799 } }
I20260807 10:55:56.655529 3067 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5505dd289b804a26a17a5a45185257e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:55:56.655700 3067 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5505dd289b804a26a17a5a45185257e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:55:56.655889 3067 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5505dd289b804a26a17a5a45185257e2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:55:56.658959 3067 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5505dd289b804a26a17a5a45185257e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5505dd289b804a26a17a5a45185257e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33799 } }
I20260807 10:55:56.659637 3067 leader_election.cc:302] T 00000000000000000000000000000000 P 5505dd289b804a26a17a5a45185257e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5505dd289b804a26a17a5a45185257e2; no voters:
I20260807 10:55:56.660460 3067 leader_election.cc:288] T 00000000000000000000000000000000 P 5505dd289b804a26a17a5a45185257e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:55:56.660648 3083 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5505dd289b804a26a17a5a45185257e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:55:56.661634 3083 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5505dd289b804a26a17a5a45185257e2 [term 1 LEADER]: Becoming Leader. State: Replica: 5505dd289b804a26a17a5a45185257e2, State: Running, Role: LEADER
I20260807 10:55:56.662513 3083 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5505dd289b804a26a17a5a45185257e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5505dd289b804a26a17a5a45185257e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33799 } }
I20260807 10:55:56.663077 3067 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5505dd289b804a26a17a5a45185257e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:55:56.667275 3084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5505dd289b804a26a17a5a45185257e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5505dd289b804a26a17a5a45185257e2. Latest consensus state: current_term: 1 leader_uuid: "5505dd289b804a26a17a5a45185257e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5505dd289b804a26a17a5a45185257e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33799 } } }
I20260807 10:55:56.667761 3084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5505dd289b804a26a17a5a45185257e2 [sys.catalog]: This master's current role is: LEADER
I20260807 10:55:56.668052 3085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5505dd289b804a26a17a5a45185257e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5505dd289b804a26a17a5a45185257e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5505dd289b804a26a17a5a45185257e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33799 } } }
I20260807 10:55:56.668463 3085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5505dd289b804a26a17a5a45185257e2 [sys.catalog]: This master's current role is: LEADER
I20260807 10:55:56.680948 3072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:56.653555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.203:37975 (local address 127.0.0.1:33799)
0807 10:55:56.653763 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:56.653769 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:56.654165 (+ 396us) server_negotiation.cc:407] Connection header received
0807 10:55:56.654312 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:56.654318 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:56.654420 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:56.654554 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:56.656567 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:56.657862 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:56.659394 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:56.660077 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:56.660248 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:56.664234 (+ 3986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:56.664245 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:56.664251 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:55:56.664313 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:56.674546 (+ 10233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:56.675836 (+ 1290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:56.675847 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:56.675858 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:56.676202 (+ 344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:55:56.677224 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:55:56.677230 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:55:56.677234 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:55:56.677912 (+ 678us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:56.679843 (+ 1931us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:56.680149 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 10:55:56.680433 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":78}
W20260807 10:55:56.684424 3107 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5505dd289b804a26a17a5a45185257e2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 10:55:56.684571 3107 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 10:55:56.684767 3090 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:55:56.687438 3090 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:55:56.698684 3090 catalog_manager.cc:1347] Generated new cluster ID: d3deaf8f5e8a4468ae8aaa8f67cfa808
I20260807 10:55:56.698840 3090 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:55:56.723999 3090 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:55:56.725556 3090 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:55:56.737329 3090 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5505dd289b804a26a17a5a45185257e2: Generated new TSK 0
I20260807 10:55:56.738550 3090 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 10:55:56.758630 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 2968
I20260807 10:55:56.760356 3061 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:55:56.863111 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 2968
W20260807 10:55:56.895526 2936 connection.cc:535] client connection to 127.0.0.1:33799 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33799): Connection reset by peer (error 104)
2026-08-07T10:55:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2590 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:55:57 dist-test-slave-x5fp krb5kdc[3114](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:55:57 dist-test-slave-x5fp krb5kdc[3114](info): set up 2 sockets
Aug 07 10:55:57 dist-test-slave-x5fp krb5kdc[3114](info): commencing operation
krb5kdc: starting...
W20260807 10:55:58.272501 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.355s user 0.006s sys 0.056s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:55:58 dist-test-slave-x5fp krb5kdc[3114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100158, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:55:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:55:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 10:55:58.673221 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34101
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34101
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:55:59.044026 3132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:55:59.044438 3132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:55:59.044631 3132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:55:59.052274 3132 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:55:59.052495 3132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:55:59.052611 3132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:55:59.052747 3132 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:55:59.052838 3132 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:55:59.063902 3132 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34101
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34101
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:55:59.065455 3132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:55:59.066859 3132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:55:59.080765 3144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:59.080886 3145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:55:59.080684 3147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:55:59.081287 3132 server_base.cc:1034] running on GCE node
I20260807 10:55:59.082720 3132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:55:59.084331 3132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:55:59.085640 3132 hybrid_clock.cc:648] HybridClock initialized: now 1786100159085577 us; error 42 us; skew 500 ppm
I20260807 10:55:59.086114 3132 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:55:59 dist-test-slave-x5fp krb5kdc[3114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100159, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:55:59.089905 3132 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 10:55:59.091889 3132 webserver.cc:460] Webserver started at http://127.0.0.1:43911/ using document root <none> and password file <none>
I20260807 10:55:59.092890 3132 fs_manager.cc:359] Metadata directory not provided
I20260807 10:55:59.093075 3132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:55:59.093461 3132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:55:59.097250 3132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "0821ae19d9bc48adbbc1c65787d4eb1e"
format_stamp: "Formatted at 2026-08-07 10:55:59 on dist-test-slave-x5fp"
server_key: "4ad7c1f0f054c96f807a59158122319c"
server_key_iv: "6c8d94784a5b5f92e00a0609a9fe2327"
server_key_version: "encryptionkey@0"
I20260807 10:55:59.098268 3132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "0821ae19d9bc48adbbc1c65787d4eb1e"
format_stamp: "Formatted at 2026-08-07 10:55:59 on dist-test-slave-x5fp"
server_key: "4ad7c1f0f054c96f807a59158122319c"
server_key_iv: "6c8d94784a5b5f92e00a0609a9fe2327"
server_key_version: "encryptionkey@0"
I20260807 10:55:59.104703 3132 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 10:55:59.109054 3156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:55:59.110642 3132 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 10:55:59.110896 3132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "0821ae19d9bc48adbbc1c65787d4eb1e"
format_stamp: "Formatted at 2026-08-07 10:55:59 on dist-test-slave-x5fp"
server_key: "4ad7c1f0f054c96f807a59158122319c"
server_key_iv: "6c8d94784a5b5f92e00a0609a9fe2327"
server_key_version: "encryptionkey@0"
I20260807 10:55:59.111184 3132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:55:59.127108 3132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:55:59.132174 3132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:55:59.132473 3132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:55:59.154475 3132 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34101
I20260807 10:55:59.154520 3226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34101 every 8 connection(s)
I20260807 10:55:59.156118 3132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:55:59.160183 3228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:55:59.162555 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 3132
I20260807 10:55:59.162781 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:55:59.163280 3367 external_mini_cluster.cc:1428] Setting key 60fdebdada7ee345aa50733fab081bb6
I20260807 10:55:59.174891 3228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0821ae19d9bc48adbbc1c65787d4eb1e: Bootstrap starting.
I20260807 10:55:59.179236 3228 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0821ae19d9bc48adbbc1c65787d4eb1e: Neither blocks nor log segments found. Creating new log.
I20260807 10:55:59.179931 3231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:59.165448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.203:41169 (local address 127.0.0.1:34101)
0807 10:55:59.166365 (+ 917us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:59.166382 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:59.166443 (+ 61us) server_negotiation.cc:407] Connection header received
0807 10:55:59.168982 (+ 2539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:59.169014 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:59.169582 (+ 568us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:59.170061 (+ 479us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:59.170630 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:59.172838 (+ 2208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:59.174236 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:59.175336 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:59.175542 (+ 206us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:59.176875 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:59.176886 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:59.176907 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:59.177668 (+ 761us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:59.177777 (+ 109us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:59.177818 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:59.177999 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:59.178270 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 10:55:59.178456 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":372,"threads_started":1}
I20260807 10:55:59.180696 3228 log.cc:824] T 00000000000000000000000000000000 P 0821ae19d9bc48adbbc1c65787d4eb1e: Log is configured to *not* fsync() on all Append() calls
I20260807 10:55:59.184247 3228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0821ae19d9bc48adbbc1c65787d4eb1e: No bootstrap required, opened a new log
I20260807 10:55:59.184443 3367 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 10:55:59.193678 3228 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0821ae19d9bc48adbbc1c65787d4eb1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0821ae19d9bc48adbbc1c65787d4eb1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34101 } }
I20260807 10:55:59.194259 3228 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0821ae19d9bc48adbbc1c65787d4eb1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:55:59.194419 3228 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0821ae19d9bc48adbbc1c65787d4eb1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0821ae19d9bc48adbbc1c65787d4eb1e, State: Initialized, Role: FOLLOWER
I20260807 10:55:59.195158 3228 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0821ae19d9bc48adbbc1c65787d4eb1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0821ae19d9bc48adbbc1c65787d4eb1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34101 } }
I20260807 10:55:59.195423 3228 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0821ae19d9bc48adbbc1c65787d4eb1e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:55:59.195597 3228 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0821ae19d9bc48adbbc1c65787d4eb1e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:55:59.195797 3228 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0821ae19d9bc48adbbc1c65787d4eb1e [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:55:59.196602 3231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:55:59.189339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.203:58321 (local address 127.0.0.1:34101)
0807 10:55:59.189554 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 10:55:59.189561 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:55:59.189893 (+ 332us) server_negotiation.cc:407] Connection header received
0807 10:55:59.190055 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:55:59.190061 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:55:59.190159 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:55:59.190287 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:55:59.190802 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:59.192061 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:59.193570 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:55:59.194249 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:55:59.194403 (+ 154us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:55:59.195759 (+ 1356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:55:59.195766 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:55:59.195771 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:55:59.195799 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:55:59.195852 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:55:59.195871 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:55:59.196013 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 10:55:59.196224 (+ 211us) server_negotiation.cc:299] Negotiation successful
0807 10:55:59.196285 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 10:55:59.199155 3228 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0821ae19d9bc48adbbc1c65787d4eb1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0821ae19d9bc48adbbc1c65787d4eb1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34101 } }
I20260807 10:55:59.199775 3228 leader_election.cc:302] T 00000000000000000000000000000000 P 0821ae19d9bc48adbbc1c65787d4eb1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0821ae19d9bc48adbbc1c65787d4eb1e; no voters:
I20260807 10:55:59.200443 3228 leader_election.cc:288] T 00000000000000000000000000000000 P 0821ae19d9bc48adbbc1c65787d4eb1e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:55:59.200641 3242 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0821ae19d9bc48adbbc1c65787d4eb1e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:55:59.201968 3242 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0821ae19d9bc48adbbc1c65787d4eb1e [term 1 LEADER]: Becoming Leader. State: Replica: 0821ae19d9bc48adbbc1c65787d4eb1e, State: Running, Role: LEADER
I20260807 10:55:59.202713 3242 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0821ae19d9bc48adbbc1c65787d4eb1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0821ae19d9bc48adbbc1c65787d4eb1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34101 } }
I20260807 10:55:59.202934 3228 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0821ae19d9bc48adbbc1c65787d4eb1e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:55:59.206203 3243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0821ae19d9bc48adbbc1c65787d4eb1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0821ae19d9bc48adbbc1c65787d4eb1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0821ae19d9bc48adbbc1c65787d4eb1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34101 } } }
I20260807 10:55:59.206666 3243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0821ae19d9bc48adbbc1c65787d4eb1e [sys.catalog]: This master's current role is: LEADER
I20260807 10:55:59.206183 3244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0821ae19d9bc48adbbc1c65787d4eb1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0821ae19d9bc48adbbc1c65787d4eb1e. Latest consensus state: current_term: 1 leader_uuid: "0821ae19d9bc48adbbc1c65787d4eb1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0821ae19d9bc48adbbc1c65787d4eb1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34101 } } }
I20260807 10:55:59.207248 3244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0821ae19d9bc48adbbc1c65787d4eb1e [sys.catalog]: This master's current role is: LEADER
I20260807 10:55:59.210067 3258 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:55:59.215919 3258 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:55:59.225466 3258 catalog_manager.cc:1347] Generated new cluster ID: 7ea43534ea7145c88873dd31d7fe93f3
I20260807 10:55:59.225605 3258 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:55:59.245585 3258 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:55:59.247025 3258 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:55:59.268812 3258 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0821ae19d9bc48adbbc1c65787d4eb1e: Generated new TSK 0
I20260807 10:55:59.269822 3258 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 10:55:59.284600 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 3132
W20260807 10:55:59.286320 3221 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.203:41169
W20260807 10:55:59.287263 3367 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 3132: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 10:55:59.287451 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 3132
W20260807 10:55:59.315521 3112 connection.cc:535] client connection to 127.0.0.1:34101 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34101): Connection reset by peer (error 104)
2026-08-07T10:55:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2418 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:55:59 dist-test-slave-x5fp krb5kdc[3275](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:55:59 dist-test-slave-x5fp krb5kdc[3275](info): set up 2 sockets
Aug 07 10:55:59 dist-test-slave-x5fp krb5kdc[3275](info): commencing operation
krb5kdc: starting...
W20260807 10:56:00.707062 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.371s user 0.000s sys 0.064s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:56:00 dist-test-slave-x5fp krb5kdc[3275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:56:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:56:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 10:56:01.109669 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33189
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33189
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:56:01.504595 3295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:56:01.505046 3295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:56:01.505224 3295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:56:01.513139 3295 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:56:01.513276 3295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:56:01.513365 3295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:56:01.513471 3295 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:56:01.513581 3295 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:56:01.524431 3295 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33189
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786099767749326-3367-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:56:01.525977 3295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:56:01.527382 3295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:56:01.540629 3301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:56:01.540954 3302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:56:01.540828 3304 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:56:01.542683 3295 server_base.cc:1034] running on GCE node
I20260807 10:56:01.543871 3295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:56:01.545804 3295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:56:01.547145 3295 hybrid_clock.cc:648] HybridClock initialized: now 1786100161547073 us; error 70 us; skew 500 ppm
I20260807 10:56:01.547583 3295 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:56:01 dist-test-slave-x5fp krb5kdc[3275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100161, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:56:01.551403 3295 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 10:56:01.553256 3295 webserver.cc:460] Webserver started at http://127.0.0.1:41975/ using document root <none> and password file <none>
I20260807 10:56:01.553980 3295 fs_manager.cc:359] Metadata directory not provided
I20260807 10:56:01.554093 3295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:56:01.554361 3295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:56:01.557755 3295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "4b5f167daa0e47189fff27d7a7caaa17"
format_stamp: "Formatted at 2026-08-07 10:56:01 on dist-test-slave-x5fp"
server_key: "62be74819b90954fc0c015b662fe7f21"
server_key_iv: "9e0885f74760b11fae4cdee67c805551"
server_key_version: "encryptionkey@0"
I20260807 10:56:01.558557 3295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "4b5f167daa0e47189fff27d7a7caaa17"
format_stamp: "Formatted at 2026-08-07 10:56:01 on dist-test-slave-x5fp"
server_key: "62be74819b90954fc0c015b662fe7f21"
server_key_iv: "9e0885f74760b11fae4cdee67c805551"
server_key_version: "encryptionkey@0"
I20260807 10:56:01.564392 3295 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260807 10:56:01.568284 3316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:56:01.569944 3295 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 10:56:01.570182 3295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "4b5f167daa0e47189fff27d7a7caaa17"
format_stamp: "Formatted at 2026-08-07 10:56:01 on dist-test-slave-x5fp"
server_key: "62be74819b90954fc0c015b662fe7f21"
server_key_iv: "9e0885f74760b11fae4cdee67c805551"
server_key_version: "encryptionkey@0"
I20260807 10:56:01.570428 3295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:56:01.577090 3295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:56:01.582197 3295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:56:01.582543 3295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:56:01.602739 3295 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33189
I20260807 10:56:01.602784 3399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33189 every 8 connection(s)
I20260807 10:56:01.604328 3295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:56:01.608641 3400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:56:01.612394 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 3295
I20260807 10:56:01.612627 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:56:01.613117 3367 external_mini_cluster.cc:1428] Setting key 48945eabb1babf65eaea3f9c48d4550b
I20260807 10:56:01.621551 3403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:01.615098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.203:41067 (local address 127.0.0.1:33189)
0807 10:56:01.615938 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:01.615951 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:01.615995 (+ 44us) server_negotiation.cc:407] Connection header received
0807 10:56:01.617772 (+ 1777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:01.617797 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:01.618210 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:01.618581 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:01.619772 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:01.619778 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:01.619792 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:56:01.619807 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:01.619872 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:56:01.619893 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:01.620121 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:01.620226 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 10:56:01.620451 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":383,"threads_started":1}
I20260807 10:56:01.622277 3400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b5f167daa0e47189fff27d7a7caaa17: Bootstrap starting.
I20260807 10:56:01.625725 3367 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 10:56:01.626093 3400 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b5f167daa0e47189fff27d7a7caaa17: Neither blocks nor log segments found. Creating new log.
I20260807 10:56:01.627164 3400 log.cc:824] T 00000000000000000000000000000000 P 4b5f167daa0e47189fff27d7a7caaa17: Log is configured to *not* fsync() on all Append() calls
I20260807 10:56:01.631117 3400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b5f167daa0e47189fff27d7a7caaa17: No bootstrap required, opened a new log
I20260807 10:56:01.633620 3403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:01.630612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.203:55627 (local address 127.0.0.1:33189)
0807 10:56:01.630856 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:01.630863 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:01.631362 (+ 499us) server_negotiation.cc:407] Connection header received
0807 10:56:01.631557 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:01.631561 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:01.631616 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:01.631713 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:01.632814 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:01.632819 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:01.632822 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:56:01.632838 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:01.632887 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:56:01.632903 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:01.633002 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:01.633175 (+ 173us) server_negotiation.cc:299] Negotiation successful
0807 10:56:01.633237 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 10:56:01.640266 3400 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b5f167daa0e47189fff27d7a7caaa17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b5f167daa0e47189fff27d7a7caaa17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33189 } }
I20260807 10:56:01.640805 3400 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b5f167daa0e47189fff27d7a7caaa17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:56:01.640929 3400 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b5f167daa0e47189fff27d7a7caaa17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b5f167daa0e47189fff27d7a7caaa17, State: Initialized, Role: FOLLOWER
I20260807 10:56:01.641650 3400 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b5f167daa0e47189fff27d7a7caaa17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b5f167daa0e47189fff27d7a7caaa17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33189 } }
I20260807 10:56:01.642009 3400 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b5f167daa0e47189fff27d7a7caaa17 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:56:01.642174 3400 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b5f167daa0e47189fff27d7a7caaa17 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:56:01.642330 3400 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b5f167daa0e47189fff27d7a7caaa17 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:56:01.644860 3400 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b5f167daa0e47189fff27d7a7caaa17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b5f167daa0e47189fff27d7a7caaa17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33189 } }
I20260807 10:56:01.645537 3400 leader_election.cc:302] T 00000000000000000000000000000000 P 4b5f167daa0e47189fff27d7a7caaa17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b5f167daa0e47189fff27d7a7caaa17; no voters:
I20260807 10:56:01.646317 3400 leader_election.cc:288] T 00000000000000000000000000000000 P 4b5f167daa0e47189fff27d7a7caaa17 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:56:01.646463 3415 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b5f167daa0e47189fff27d7a7caaa17 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:56:01.647248 3415 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b5f167daa0e47189fff27d7a7caaa17 [term 1 LEADER]: Becoming Leader. State: Replica: 4b5f167daa0e47189fff27d7a7caaa17, State: Running, Role: LEADER
I20260807 10:56:01.647877 3415 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b5f167daa0e47189fff27d7a7caaa17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b5f167daa0e47189fff27d7a7caaa17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33189 } }
I20260807 10:56:01.648703 3400 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b5f167daa0e47189fff27d7a7caaa17 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:56:01.651660 3416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b5f167daa0e47189fff27d7a7caaa17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b5f167daa0e47189fff27d7a7caaa17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b5f167daa0e47189fff27d7a7caaa17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33189 } } }
I20260807 10:56:01.651731 3417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b5f167daa0e47189fff27d7a7caaa17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b5f167daa0e47189fff27d7a7caaa17. Latest consensus state: current_term: 1 leader_uuid: "4b5f167daa0e47189fff27d7a7caaa17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b5f167daa0e47189fff27d7a7caaa17" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33189 } } }
I20260807 10:56:01.652144 3416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b5f167daa0e47189fff27d7a7caaa17 [sys.catalog]: This master's current role is: LEADER
I20260807 10:56:01.652154 3417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b5f167daa0e47189fff27d7a7caaa17 [sys.catalog]: This master's current role is: LEADER
I20260807 10:56:01.654450 3426 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:56:01.660068 3426 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:56:01.670691 3426 catalog_manager.cc:1347] Generated new cluster ID: bec5180e3faf42f2a62948940fd76e2d
I20260807 10:56:01.670831 3426 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:56:01.690984 3426 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:56:01.691506 3426 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:56:01.704142 3426 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b5f167daa0e47189fff27d7a7caaa17: Generated new TSK 0
I20260807 10:56:01.705258 3426 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 10:56:01.713315 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 3295
W20260807 10:56:01.714552 3395 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.203:41067
W20260807 10:56:01.715361 3367 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 3295: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 10:56:01.715561 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 3295
2026-08-07T10:56:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2422 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:56:01 dist-test-slave-x5fp krb5kdc[3447](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:56:01 dist-test-slave-x5fp krb5kdc[3447](info): set up 2 sockets
Aug 07 10:56:01 dist-test-slave-x5fp krb5kdc[3447](info): commencing operation
krb5kdc: starting...
W20260807 10:56:03.126273 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.368s user 0.005s sys 0.060s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:56:03 dist-test-slave-x5fp krb5kdc[3447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:56:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:56:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.73.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
I20260807 10:56:03.554095 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:40631
--webserver_interface=127.3.73.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:43899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:40631
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:56:03.935655 3469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:56:03.935994 3469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:56:03.936197 3469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:56:03.943957 3469 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:56:03.944098 3469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:56:03.944198 3469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:56:03.944321 3469 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:56:03.944406 3469 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:56:03.958388 3469 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:43899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:40631
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:40631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:56:03.960491 3469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:56:03.962483 3469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:56:03.979560 3481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:56:03.979665 3479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:56:03.979607 3477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:56:03.982920 3469 server_base.cc:1034] running on GCE node
I20260807 10:56:03.984032 3469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:56:03.986086 3469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:56:03.987391 3469 hybrid_clock.cc:648] HybridClock initialized: now 1786100163987335 us; error 50 us; skew 500 ppm
I20260807 10:56:03.987869 3469 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:56:03 dist-test-slave-x5fp krb5kdc[3447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100163, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:56:03.992223 3469 init.cc:377] Logged in from keytab as kudu/127.3.73.254@KRBTEST.COM (short username kudu)
I20260807 10:56:03.994596 3469 webserver.cc:460] Webserver started at http://127.3.73.254:45577/ using document root <none> and password file <none>
I20260807 10:56:03.995494 3469 fs_manager.cc:359] Metadata directory not provided
I20260807 10:56:03.995674 3469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:56:03.996047 3469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:56:03.999702 3469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "b0dbe57b8bab4030963ce74687bc2690"
format_stamp: "Formatted at 2026-08-07 10:56:03 on dist-test-slave-x5fp"
server_key: "739834a36264b0df172bee92d7c299a3"
server_key_iv: "4e79d3aa625c1229285a481e8727cef0"
server_key_version: "encryptionkey@0"
I20260807 10:56:04.000838 3469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "b0dbe57b8bab4030963ce74687bc2690"
format_stamp: "Formatted at 2026-08-07 10:56:03 on dist-test-slave-x5fp"
server_key: "739834a36264b0df172bee92d7c299a3"
server_key_iv: "4e79d3aa625c1229285a481e8727cef0"
server_key_version: "encryptionkey@0"
I20260807 10:56:04.007468 3469 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260807 10:56:04.012204 3490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:56:04.014006 3469 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 10:56:04.014292 3469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "b0dbe57b8bab4030963ce74687bc2690"
format_stamp: "Formatted at 2026-08-07 10:56:03 on dist-test-slave-x5fp"
server_key: "739834a36264b0df172bee92d7c299a3"
server_key_iv: "4e79d3aa625c1229285a481e8727cef0"
server_key_version: "encryptionkey@0"
I20260807 10:56:04.014576 3469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:56:04.037313 3469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:56:04.042613 3469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:56:04.043018 3469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:56:04.064098 3469 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:40631
I20260807 10:56:04.064141 3558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:40631 every 8 connection(s)
I20260807 10:56:04.065744 3469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:56:04.068149 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 3469
I20260807 10:56:04.068401 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:56:04.069063 3367 external_mini_cluster.cc:1428] Setting key 59b21e89484e9af53d01c4b8fde8b389
I20260807 10:56:04.071589 3561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:56:04.084208 3561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0dbe57b8bab4030963ce74687bc2690: Bootstrap starting.
Aug 07 10:56:04 dist-test-slave-x5fp krb5kdc[3447](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:56:04.088475 3561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0dbe57b8bab4030963ce74687bc2690: Neither blocks nor log segments found. Creating new log.
I20260807 10:56:04.089737 3561 log.cc:824] T 00000000000000000000000000000000 P b0dbe57b8bab4030963ce74687bc2690: Log is configured to *not* fsync() on all Append() calls
I20260807 10:56:04.093493 3561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0dbe57b8bab4030963ce74687bc2690: No bootstrap required, opened a new log
I20260807 10:56:04.096702 3565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:04.071264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49997 (local address 127.3.73.254:40631)
0807 10:56:04.072618 (+ 1354us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:04.072631 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:04.072676 (+ 45us) server_negotiation.cc:407] Connection header received
0807 10:56:04.074549 (+ 1873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:04.074575 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:04.075062 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:04.075551 (+ 489us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:04.077263 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:04.078660 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:04.080011 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:04.080820 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:04.081032 (+ 212us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:04.085959 (+ 4927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:04.085969 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:04.085986 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:56:04.086037 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:04.090210 (+ 4173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:04.091433 (+ 1223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:04.091441 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:04.091448 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:04.091655 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:04.092337 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:04.092341 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:04.092344 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:04.093100 (+ 756us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:04.093362 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:04.094638 (+ 1276us) server_negotiation.cc:299] Negotiation successful
0807 10:56:04.095060 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1122,"thread_start_us":525,"threads_started":1}
I20260807 10:56:04.101508 3367 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 10:56:04.103869 3561 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0dbe57b8bab4030963ce74687bc2690 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0dbe57b8bab4030963ce74687bc2690" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 40631 } }
I20260807 10:56:04.104472 3561 raft_consensus.cc:374] T 00000000000000000000000000000000 P b0dbe57b8bab4030963ce74687bc2690 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:56:04.104609 3561 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0dbe57b8bab4030963ce74687bc2690 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0dbe57b8bab4030963ce74687bc2690, State: Initialized, Role: FOLLOWER
I20260807 10:56:04.105548 3561 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0dbe57b8bab4030963ce74687bc2690 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0dbe57b8bab4030963ce74687bc2690" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 40631 } }
I20260807 10:56:04.105928 3561 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0dbe57b8bab4030963ce74687bc2690 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:56:04.106130 3561 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0dbe57b8bab4030963ce74687bc2690 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:56:04.106339 3561 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0dbe57b8bab4030963ce74687bc2690 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:56:04.109457 3561 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0dbe57b8bab4030963ce74687bc2690 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0dbe57b8bab4030963ce74687bc2690" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 40631 } }
I20260807 10:56:04.110132 3561 leader_election.cc:302] T 00000000000000000000000000000000 P b0dbe57b8bab4030963ce74687bc2690 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0dbe57b8bab4030963ce74687bc2690; no voters:
I20260807 10:56:04.110880 3561 leader_election.cc:288] T 00000000000000000000000000000000 P b0dbe57b8bab4030963ce74687bc2690 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:56:04.111040 3574 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0dbe57b8bab4030963ce74687bc2690 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:56:04.114085 3561 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0dbe57b8bab4030963ce74687bc2690 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:56:04.116619 3574 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0dbe57b8bab4030963ce74687bc2690 [term 1 LEADER]: Becoming Leader. State: Replica: b0dbe57b8bab4030963ce74687bc2690, State: Running, Role: LEADER
I20260807 10:56:04.117694 3574 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0dbe57b8bab4030963ce74687bc2690 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0dbe57b8bab4030963ce74687bc2690" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 40631 } }
I20260807 10:56:04.122452 3576 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0dbe57b8bab4030963ce74687bc2690 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b0dbe57b8bab4030963ce74687bc2690" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0dbe57b8bab4030963ce74687bc2690" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 40631 } } }
I20260807 10:56:04.122999 3576 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0dbe57b8bab4030963ce74687bc2690 [sys.catalog]: This master's current role is: LEADER
I20260807 10:56:04.122715 3577 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0dbe57b8bab4030963ce74687bc2690 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0dbe57b8bab4030963ce74687bc2690. Latest consensus state: current_term: 1 leader_uuid: "b0dbe57b8bab4030963ce74687bc2690" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0dbe57b8bab4030963ce74687bc2690" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 40631 } } }
I20260807 10:56:04.123143 3577 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0dbe57b8bab4030963ce74687bc2690 [sys.catalog]: This master's current role is: LEADER
I20260807 10:56:04.127884 3585 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:56:04.129660 3565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:04.107554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39809 (local address 127.3.73.254:40631)
0807 10:56:04.107770 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:04.107777 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:04.108405 (+ 628us) server_negotiation.cc:407] Connection header received
0807 10:56:04.108596 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:04.108602 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:04.108687 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:04.108844 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:04.110718 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:04.112290 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:04.113965 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:04.114606 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:04.114791 (+ 185us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:04.118673 (+ 3882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:04.118679 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:04.118683 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:56:04.118758 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:04.124610 (+ 5852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:04.126114 (+ 1504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:04.126122 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:04.126128 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:04.126337 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:04.127241 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:04.127248 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:04.127254 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:04.127846 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:04.128400 (+ 554us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:04.128914 (+ 514us) server_negotiation.cc:299] Negotiation successful
0807 10:56:04.129172 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 10:56:04.134399 3585 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:56:04.145330 3585 catalog_manager.cc:1347] Generated new cluster ID: 8a7afc824dda40b8904053d6b6eeb260
I20260807 10:56:04.145538 3585 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:56:04.164076 3585 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:56:04.165797 3585 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:56:04.180047 3585 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b0dbe57b8bab4030963ce74687bc2690: Generated new TSK 0
I20260807 10:56:04.181339 3585 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 10:56:04.196792 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 3469
I20260807 10:56:04.198367 3553 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:56:04.294307 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 3469
W20260807 10:56:04.321972 3445 connection.cc:535] client connection to 127.3.73.254:40631 recv error: Network error: failed to read from TLS socket (remote: 127.3.73.254:40631): Connection reset by peer (error 104)
2026-08-07T10:56:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2584 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:56:04 dist-test-slave-x5fp krb5kdc[3612](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:56:04 dist-test-slave-x5fp krb5kdc[3612](info): set up 2 sockets
Aug 07 10:56:04 dist-test-slave-x5fp krb5kdc[3612](info): commencing operation
krb5kdc: starting...
W20260807 10:56:05.706337 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.364s user 0.000s sys 0.067s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:56:05 dist-test-slave-x5fp krb5kdc[3612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:56:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:56:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.73.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
I20260807 10:56:06.130133 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:44867
--webserver_interface=127.3.73.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:41573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:44867
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:56:06.519877 3635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:56:06.520326 3635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:56:06.520509 3635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:56:06.528290 3635 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:56:06.528486 3635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:56:06.528607 3635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:56:06.528721 3635 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:56:06.528848 3635 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:56:06.539460 3635 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:41573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:44867
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:44867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:56:06.541199 3635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:56:06.543231 3635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:56:06.557301 3642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:56:06.557348 3644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:56:06.557325 3646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:56:06.559059 3635 server_base.cc:1034] running on GCE node
I20260807 10:56:06.560084 3635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:56:06.561908 3635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:56:06.563197 3635 hybrid_clock.cc:648] HybridClock initialized: now 1786100166563126 us; error 64 us; skew 500 ppm
I20260807 10:56:06.563633 3635 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:56:06 dist-test-slave-x5fp krb5kdc[3612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100166, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:56:06.567422 3635 init.cc:377] Logged in from keytab as kudu/127.3.73.254@KRBTEST.COM (short username kudu)
I20260807 10:56:06.569279 3635 webserver.cc:460] Webserver started at http://127.3.73.254:37867/ using document root <none> and password file <none>
I20260807 10:56:06.570093 3635 fs_manager.cc:359] Metadata directory not provided
I20260807 10:56:06.570210 3635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:56:06.570493 3635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:56:06.574067 3635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "45a043a05f2c4521b58e88bd384a769e"
format_stamp: "Formatted at 2026-08-07 10:56:06 on dist-test-slave-x5fp"
server_key: "68efdd65201b6e20a45a3e60c3b6e19d"
server_key_iv: "920db8ab3ba918ab3679f314b760d77b"
server_key_version: "encryptionkey@0"
I20260807 10:56:06.574954 3635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "45a043a05f2c4521b58e88bd384a769e"
format_stamp: "Formatted at 2026-08-07 10:56:06 on dist-test-slave-x5fp"
server_key: "68efdd65201b6e20a45a3e60c3b6e19d"
server_key_iv: "920db8ab3ba918ab3679f314b760d77b"
server_key_version: "encryptionkey@0"
I20260807 10:56:06.581277 3635 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260807 10:56:06.585451 3655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:56:06.587137 3635 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 10:56:06.587401 3635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "45a043a05f2c4521b58e88bd384a769e"
format_stamp: "Formatted at 2026-08-07 10:56:06 on dist-test-slave-x5fp"
server_key: "68efdd65201b6e20a45a3e60c3b6e19d"
server_key_iv: "920db8ab3ba918ab3679f314b760d77b"
server_key_version: "encryptionkey@0"
I20260807 10:56:06.587659 3635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:56:06.603663 3635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:56:06.609125 3635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:56:06.609504 3635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:56:06.631366 3635 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:44867
I20260807 10:56:06.631423 3723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:44867 every 8 connection(s)
I20260807 10:56:06.633172 3635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:56:06.637552 3724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:56:06.641217 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 3635
I20260807 10:56:06.641412 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:56:06.641906 3367 external_mini_cluster.cc:1428] Setting key 42c5f74f0a31440a8e70144ae99ccbb7
I20260807 10:56:06.650468 3724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45a043a05f2c4521b58e88bd384a769e: Bootstrap starting.
I20260807 10:56:06.653962 3724 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45a043a05f2c4521b58e88bd384a769e: Neither blocks nor log segments found. Creating new log.
I20260807 10:56:06.655283 3724 log.cc:824] T 00000000000000000000000000000000 P 45a043a05f2c4521b58e88bd384a769e: Log is configured to *not* fsync() on all Append() calls
I20260807 10:56:06.656383 3727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:06.643923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60645 (local address 127.3.73.254:44867)
0807 10:56:06.644904 (+ 981us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:06.644916 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:06.644961 (+ 45us) server_negotiation.cc:407] Connection header received
0807 10:56:06.646739 (+ 1778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:06.646766 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:06.647234 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:06.647668 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:06.648298 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:06.649680 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:06.651162 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:06.651859 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:06.652101 (+ 242us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:06.653504 (+ 1403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:06.653513 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:06.653528 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:56:06.654134 (+ 606us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:06.654234 (+ 100us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:56:06.654264 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:06.654462 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:06.654767 (+ 305us) server_negotiation.cc:299] Negotiation successful
0807 10:56:06.654956 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":399,"threads_started":1}
I20260807 10:56:06.658860 3724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45a043a05f2c4521b58e88bd384a769e: No bootstrap required, opened a new log
I20260807 10:56:06.660940 3367 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 10:56:06.667789 3724 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45a043a05f2c4521b58e88bd384a769e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45a043a05f2c4521b58e88bd384a769e" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 44867 } }
I20260807 10:56:06.668365 3724 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45a043a05f2c4521b58e88bd384a769e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:56:06.668476 3724 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45a043a05f2c4521b58e88bd384a769e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45a043a05f2c4521b58e88bd384a769e, State: Initialized, Role: FOLLOWER
I20260807 10:56:06.669216 3724 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45a043a05f2c4521b58e88bd384a769e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45a043a05f2c4521b58e88bd384a769e" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 44867 } }
I20260807 10:56:06.669507 3724 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45a043a05f2c4521b58e88bd384a769e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:56:06.669664 3724 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45a043a05f2c4521b58e88bd384a769e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:56:06.669864 3724 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45a043a05f2c4521b58e88bd384a769e [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:56:06.673404 3727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:06.665771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39295 (local address 127.3.73.254:44867)
0807 10:56:06.666003 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:06.666011 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:06.666491 (+ 480us) server_negotiation.cc:407] Connection header received
0807 10:56:06.666660 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:06.666666 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:06.666749 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:06.666879 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:06.667451 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:06.668615 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:06.670286 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:06.670972 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:06.671120 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:06.672383 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:06.672390 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:06.672395 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:56:06.672432 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:06.672483 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:56:06.672505 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:06.672610 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:06.672930 (+ 320us) server_negotiation.cc:299] Negotiation successful
0807 10:56:06.673001 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260807 10:56:06.673403 3724 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45a043a05f2c4521b58e88bd384a769e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45a043a05f2c4521b58e88bd384a769e" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 44867 } }
I20260807 10:56:06.674046 3724 leader_election.cc:302] T 00000000000000000000000000000000 P 45a043a05f2c4521b58e88bd384a769e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45a043a05f2c4521b58e88bd384a769e; no voters:
I20260807 10:56:06.674794 3724 leader_election.cc:288] T 00000000000000000000000000000000 P 45a043a05f2c4521b58e88bd384a769e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:56:06.674973 3740 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45a043a05f2c4521b58e88bd384a769e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:56:06.675807 3740 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45a043a05f2c4521b58e88bd384a769e [term 1 LEADER]: Becoming Leader. State: Replica: 45a043a05f2c4521b58e88bd384a769e, State: Running, Role: LEADER
I20260807 10:56:06.676426 3740 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45a043a05f2c4521b58e88bd384a769e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45a043a05f2c4521b58e88bd384a769e" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 44867 } }
I20260807 10:56:06.677279 3724 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45a043a05f2c4521b58e88bd384a769e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:56:06.679315 3741 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45a043a05f2c4521b58e88bd384a769e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45a043a05f2c4521b58e88bd384a769e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45a043a05f2c4521b58e88bd384a769e" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 44867 } } }
I20260807 10:56:06.679718 3741 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45a043a05f2c4521b58e88bd384a769e [sys.catalog]: This master's current role is: LEADER
I20260807 10:56:06.684538 3742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45a043a05f2c4521b58e88bd384a769e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45a043a05f2c4521b58e88bd384a769e. Latest consensus state: current_term: 1 leader_uuid: "45a043a05f2c4521b58e88bd384a769e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45a043a05f2c4521b58e88bd384a769e" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 44867 } } }
I20260807 10:56:06.684969 3742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45a043a05f2c4521b58e88bd384a769e [sys.catalog]: This master's current role is: LEADER
I20260807 10:56:06.686172 3748 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:56:06.692896 3748 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:56:06.703051 3748 catalog_manager.cc:1347] Generated new cluster ID: 3a8f063c77a542aaac00870617fa5bb0
I20260807 10:56:06.703210 3748 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:56:06.726552 3748 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:56:06.728680 3748 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:56:06.738111 3748 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45a043a05f2c4521b58e88bd384a769e: Generated new TSK 0
I20260807 10:56:06.739338 3748 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 10:56:06.758443 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 3635
W20260807 10:56:06.760205 3719 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:60645
W20260807 10:56:06.761178 3367 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 3635: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 10:56:06.761359 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 3635
W20260807 10:56:06.791384 3610 connection.cc:535] client connection to 127.3.73.254:44867 recv error: Network error: failed to read from TLS socket (remote: 127.3.73.254:44867): Connection reset by peer (error 104)
2026-08-07T10:56:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2468 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:56:06 dist-test-slave-x5fp krb5kdc[3782](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:56:06 dist-test-slave-x5fp krb5kdc[3782](info): set up 2 sockets
Aug 07 10:56:06 dist-test-slave-x5fp krb5kdc[3782](info): commencing operation
krb5kdc: starting...
W20260807 10:56:08.177086 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.366s user 0.006s sys 0.059s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:56:08 dist-test-slave-x5fp krb5kdc[3782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:56:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:56:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.73.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
I20260807 10:56:08.593693 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:42547
--webserver_interface=127.3.73.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:39831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:42547
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:56:08.970201 3804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:56:08.970525 3804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:56:08.970696 3804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:56:08.978354 3804 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:56:08.978494 3804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:56:08.978617 3804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:56:08.978713 3804 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:56:08.978816 3804 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:56:08.989426 3804 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:39831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:42547
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:42547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:56:08.990923 3804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:56:08.992388 3804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:56:09.005203 3811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:56:09.005414 3812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:56:09.005545 3814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:56:09.006422 3804 server_base.cc:1034] running on GCE node
I20260807 10:56:09.007262 3804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:56:09.008834 3804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:56:09.010108 3804 hybrid_clock.cc:648] HybridClock initialized: now 1786100169010000 us; error 62 us; skew 500 ppm
I20260807 10:56:09.010519 3804 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:56:09 dist-test-slave-x5fp krb5kdc[3782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100169, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:56:09.014123 3804 init.cc:377] Logged in from keytab as kudu/127.3.73.254@KRBTEST.COM (short username kudu)
I20260807 10:56:09.015837 3804 webserver.cc:460] Webserver started at http://127.3.73.254:34725/ using document root <none> and password file <none>
I20260807 10:56:09.016628 3804 fs_manager.cc:359] Metadata directory not provided
I20260807 10:56:09.016811 3804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:56:09.017203 3804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:56:09.020474 3804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "1a2adff50f6540088aad9c76d40401f8"
format_stamp: "Formatted at 2026-08-07 10:56:09 on dist-test-slave-x5fp"
server_key: "e76b742f6e19c2266c7c73607a733f80"
server_key_iv: "23826b6033f8b54cbdf4cb1d62fabde0"
server_key_version: "encryptionkey@0"
I20260807 10:56:09.021443 3804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "1a2adff50f6540088aad9c76d40401f8"
format_stamp: "Formatted at 2026-08-07 10:56:09 on dist-test-slave-x5fp"
server_key: "e76b742f6e19c2266c7c73607a733f80"
server_key_iv: "23826b6033f8b54cbdf4cb1d62fabde0"
server_key_version: "encryptionkey@0"
I20260807 10:56:09.027626 3804 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260807 10:56:09.031520 3825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:56:09.033151 3804 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 10:56:09.033355 3804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "1a2adff50f6540088aad9c76d40401f8"
format_stamp: "Formatted at 2026-08-07 10:56:09 on dist-test-slave-x5fp"
server_key: "e76b742f6e19c2266c7c73607a733f80"
server_key_iv: "23826b6033f8b54cbdf4cb1d62fabde0"
server_key_version: "encryptionkey@0"
I20260807 10:56:09.033591 3804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:56:09.039839 3804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:56:09.044984 3804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:56:09.045336 3804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:56:09.066439 3804 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:42547
I20260807 10:56:09.066479 3906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:42547 every 8 connection(s)
I20260807 10:56:09.067966 3804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:56:09.072222 3907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:56:09.074112 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 3804
I20260807 10:56:09.074311 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:56:09.074903 3367 external_mini_cluster.cc:1428] Setting key cd415e054433e80c4656594a505915aa
I20260807 10:56:09.083503 3910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:09.076979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45925 (local address 127.3.73.254:42547)
0807 10:56:09.077975 (+ 996us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:09.077988 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:09.078040 (+ 52us) server_negotiation.cc:407] Connection header received
0807 10:56:09.079702 (+ 1662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:09.079742 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:09.080146 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:09.080621 (+ 475us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:09.081772 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:09.081779 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:09.081793 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:56:09.081807 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:09.081873 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:56:09.081894 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:09.082148 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:09.082298 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 10:56:09.082501 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":694,"thread_start_us":424,"threads_started":1}
I20260807 10:56:09.087042 3907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a2adff50f6540088aad9c76d40401f8: Bootstrap starting.
I20260807 10:56:09.087667 3367 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 10:56:09.090547 3907 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a2adff50f6540088aad9c76d40401f8: Neither blocks nor log segments found. Creating new log.
I20260807 10:56:09.091684 3907 log.cc:824] T 00000000000000000000000000000000 P 1a2adff50f6540088aad9c76d40401f8: Log is configured to *not* fsync() on all Append() calls
I20260807 10:56:09.095284 3910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:09.092396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47911 (local address 127.3.73.254:42547)
0807 10:56:09.092667 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:09.092675 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:09.093119 (+ 444us) server_negotiation.cc:407] Connection header received
0807 10:56:09.093327 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:09.093332 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:09.093392 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:09.093513 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:09.094581 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:09.094586 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:09.094590 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:56:09.094608 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:09.094657 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:56:09.094677 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:09.094814 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:09.094927 (+ 113us) server_negotiation.cc:299] Negotiation successful
0807 10:56:09.094990 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260807 10:56:09.095503 3907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a2adff50f6540088aad9c76d40401f8: No bootstrap required, opened a new log
I20260807 10:56:09.103958 3907 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a2adff50f6540088aad9c76d40401f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a2adff50f6540088aad9c76d40401f8" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 42547 } }
I20260807 10:56:09.104444 3907 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a2adff50f6540088aad9c76d40401f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:56:09.104586 3907 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a2adff50f6540088aad9c76d40401f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a2adff50f6540088aad9c76d40401f8, State: Initialized, Role: FOLLOWER
I20260807 10:56:09.105257 3907 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a2adff50f6540088aad9c76d40401f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a2adff50f6540088aad9c76d40401f8" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 42547 } }
I20260807 10:56:09.105518 3907 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a2adff50f6540088aad9c76d40401f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:56:09.105703 3907 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a2adff50f6540088aad9c76d40401f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:56:09.105890 3907 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a2adff50f6540088aad9c76d40401f8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:56:09.108333 3907 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a2adff50f6540088aad9c76d40401f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a2adff50f6540088aad9c76d40401f8" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 42547 } }
I20260807 10:56:09.108985 3907 leader_election.cc:302] T 00000000000000000000000000000000 P 1a2adff50f6540088aad9c76d40401f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a2adff50f6540088aad9c76d40401f8; no voters:
I20260807 10:56:09.109814 3907 leader_election.cc:288] T 00000000000000000000000000000000 P 1a2adff50f6540088aad9c76d40401f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:56:09.110033 3917 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a2adff50f6540088aad9c76d40401f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:56:09.111383 3917 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a2adff50f6540088aad9c76d40401f8 [term 1 LEADER]: Becoming Leader. State: Replica: 1a2adff50f6540088aad9c76d40401f8, State: Running, Role: LEADER
I20260807 10:56:09.112027 3917 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a2adff50f6540088aad9c76d40401f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a2adff50f6540088aad9c76d40401f8" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 42547 } }
I20260807 10:56:09.112332 3907 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a2adff50f6540088aad9c76d40401f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:56:09.115325 3919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a2adff50f6540088aad9c76d40401f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a2adff50f6540088aad9c76d40401f8. Latest consensus state: current_term: 1 leader_uuid: "1a2adff50f6540088aad9c76d40401f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a2adff50f6540088aad9c76d40401f8" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 42547 } } }
I20260807 10:56:09.115860 3919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a2adff50f6540088aad9c76d40401f8 [sys.catalog]: This master's current role is: LEADER
I20260807 10:56:09.115381 3918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a2adff50f6540088aad9c76d40401f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a2adff50f6540088aad9c76d40401f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a2adff50f6540088aad9c76d40401f8" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 42547 } } }
I20260807 10:56:09.116209 3918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a2adff50f6540088aad9c76d40401f8 [sys.catalog]: This master's current role is: LEADER
I20260807 10:56:09.117122 3927 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:56:09.123720 3927 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:56:09.134403 3927 catalog_manager.cc:1347] Generated new cluster ID: f0a64ac6fc35478c91dbe2ff1209da3c
I20260807 10:56:09.134564 3927 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:56:09.147919 3927 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:56:09.148478 3927 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:56:09.162032 3927 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a2adff50f6540088aad9c76d40401f8: Generated new TSK 0
I20260807 10:56:09.163244 3927 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 10:56:09.171983 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 3804
W20260807 10:56:09.173396 3898 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45925
W20260807 10:56:09.174263 3367 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 3804: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 10:56:09.174438 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 3804
2026-08-07T10:56:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2412 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:56:09 dist-test-slave-x5fp krb5kdc[3943](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:56:09 dist-test-slave-x5fp krb5kdc[3943](info): set up 2 sockets
Aug 07 10:56:09 dist-test-slave-x5fp krb5kdc[3943](info): commencing operation
krb5kdc: starting...
W20260807 10:56:10.585062 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.361s user 0.006s sys 0.059s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:56:10 dist-test-slave-x5fp krb5kdc[3943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:56:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:56:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.73.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
I20260807 10:56:10.997653 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:44753
--webserver_interface=127.3.73.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:33935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:44753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:56:11.394853 3964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:56:11.395221 3964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:56:11.395398 3964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:56:11.403227 3964 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:56:11.403378 3964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:56:11.403487 3964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:56:11.403616 3964 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:56:11.403684 3964 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:56:11.414356 3964 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:33935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:44753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:44753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:56:11.415906 3964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:56:11.417415 3964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:56:11.431021 3970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:56:11.430989 3975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:56:11.430963 3972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:56:11.431933 3964 server_base.cc:1034] running on GCE node
I20260807 10:56:11.432986 3964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:56:11.434718 3964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:56:11.436162 3964 hybrid_clock.cc:648] HybridClock initialized: now 1786100171436106 us; error 96 us; skew 500 ppm
I20260807 10:56:11.436635 3964 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:56:11 dist-test-slave-x5fp krb5kdc[3943](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100171, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:56:11.445353 3964 init.cc:377] Logged in from keytab as kudu/127.3.73.254@KRBTEST.COM (short username kudu)
I20260807 10:56:11.447366 3964 webserver.cc:460] Webserver started at http://127.3.73.254:43103/ using document root <none> and password file <none>
I20260807 10:56:11.448314 3964 fs_manager.cc:359] Metadata directory not provided
I20260807 10:56:11.448522 3964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:56:11.448985 3964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:56:11.452673 3964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "32edabcfe95f423cacb1ffea938cd40f"
format_stamp: "Formatted at 2026-08-07 10:56:11 on dist-test-slave-x5fp"
server_key: "a4e12041326f7e9e3c378bb038055537"
server_key_iv: "d308256965b63fc014d814a11b30e72b"
server_key_version: "encryptionkey@0"
I20260807 10:56:11.453715 3964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "32edabcfe95f423cacb1ffea938cd40f"
format_stamp: "Formatted at 2026-08-07 10:56:11 on dist-test-slave-x5fp"
server_key: "a4e12041326f7e9e3c378bb038055537"
server_key_iv: "d308256965b63fc014d814a11b30e72b"
server_key_version: "encryptionkey@0"
I20260807 10:56:11.460597 3964 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260807 10:56:11.465189 3985 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:56:11.466941 3964 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260807 10:56:11.467199 3964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "32edabcfe95f423cacb1ffea938cd40f"
format_stamp: "Formatted at 2026-08-07 10:56:11 on dist-test-slave-x5fp"
server_key: "a4e12041326f7e9e3c378bb038055537"
server_key_iv: "d308256965b63fc014d814a11b30e72b"
server_key_version: "encryptionkey@0"
I20260807 10:56:11.467466 3964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:56:11.488054 3964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:56:11.493645 3964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:56:11.494009 3964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:56:11.514976 3964 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:44753
I20260807 10:56:11.515010 4047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:44753 every 8 connection(s)
I20260807 10:56:11.516686 3964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:56:11.520303 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 3964
I20260807 10:56:11.520510 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:56:11.521133 3367 external_mini_cluster.cc:1428] Setting key 8ecb0a6b184554b4161da19a122f7f1d
I20260807 10:56:11.521262 4048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:56:11.537612 4048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32edabcfe95f423cacb1ffea938cd40f: Bootstrap starting.
Aug 07 10:56:11 dist-test-slave-x5fp krb5kdc[3943](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:56:11.542516 4048 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32edabcfe95f423cacb1ffea938cd40f: Neither blocks nor log segments found. Creating new log.
I20260807 10:56:11.544075 4048 log.cc:824] T 00000000000000000000000000000000 P 32edabcfe95f423cacb1ffea938cd40f: Log is configured to *not* fsync() on all Append() calls
I20260807 10:56:11.548574 4048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32edabcfe95f423cacb1ffea938cd40f: No bootstrap required, opened a new log
I20260807 10:56:11.551370 4053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:11.523214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37757 (local address 127.3.73.254:44753)
0807 10:56:11.524203 (+ 989us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:11.524222 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:11.524292 (+ 70us) server_negotiation.cc:407] Connection header received
0807 10:56:11.526876 (+ 2584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:11.526914 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:11.527573 (+ 659us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:11.528111 (+ 538us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:11.529901 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:11.531860 (+ 1959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:11.533570 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:11.534778 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:11.534991 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:11.540655 (+ 5664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:11.540668 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:11.540685 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:56:11.540829 (+ 144us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:11.545924 (+ 5095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:11.547153 (+ 1229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:11.547162 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:11.547168 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:11.547410 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:11.547947 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:11.547952 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:11.547955 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:11.548888 (+ 933us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:11.549170 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:11.549757 (+ 587us) server_negotiation.cc:299] Negotiation successful
0807 10:56:11.550207 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":720,"thread_start_us":369,"threads_started":1}
I20260807 10:56:11.556263 3367 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 10:56:11.559150 4048 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32edabcfe95f423cacb1ffea938cd40f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32edabcfe95f423cacb1ffea938cd40f" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 44753 } }
I20260807 10:56:11.559651 4048 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32edabcfe95f423cacb1ffea938cd40f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:56:11.559785 4048 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32edabcfe95f423cacb1ffea938cd40f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32edabcfe95f423cacb1ffea938cd40f, State: Initialized, Role: FOLLOWER
I20260807 10:56:11.560488 4048 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32edabcfe95f423cacb1ffea938cd40f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32edabcfe95f423cacb1ffea938cd40f" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 44753 } }
I20260807 10:56:11.560827 4048 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32edabcfe95f423cacb1ffea938cd40f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:56:11.560992 4048 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32edabcfe95f423cacb1ffea938cd40f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:56:11.561215 4048 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32edabcfe95f423cacb1ffea938cd40f [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:56:11.565006 4048 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32edabcfe95f423cacb1ffea938cd40f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32edabcfe95f423cacb1ffea938cd40f" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 44753 } }
I20260807 10:56:11.565754 4048 leader_election.cc:302] T 00000000000000000000000000000000 P 32edabcfe95f423cacb1ffea938cd40f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32edabcfe95f423cacb1ffea938cd40f; no voters:
I20260807 10:56:11.566512 4048 leader_election.cc:288] T 00000000000000000000000000000000 P 32edabcfe95f423cacb1ffea938cd40f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:56:11.566659 4064 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32edabcfe95f423cacb1ffea938cd40f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:56:11.568277 4064 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32edabcfe95f423cacb1ffea938cd40f [term 1 LEADER]: Becoming Leader. State: Replica: 32edabcfe95f423cacb1ffea938cd40f, State: Running, Role: LEADER
I20260807 10:56:11.569012 4064 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32edabcfe95f423cacb1ffea938cd40f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32edabcfe95f423cacb1ffea938cd40f" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 44753 } }
I20260807 10:56:11.569245 4048 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32edabcfe95f423cacb1ffea938cd40f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:56:11.572495 4067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32edabcfe95f423cacb1ffea938cd40f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "32edabcfe95f423cacb1ffea938cd40f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32edabcfe95f423cacb1ffea938cd40f" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 44753 } } }
I20260807 10:56:11.572979 4067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32edabcfe95f423cacb1ffea938cd40f [sys.catalog]: This master's current role is: LEADER
I20260807 10:56:11.573467 4069 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32edabcfe95f423cacb1ffea938cd40f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32edabcfe95f423cacb1ffea938cd40f. Latest consensus state: current_term: 1 leader_uuid: "32edabcfe95f423cacb1ffea938cd40f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32edabcfe95f423cacb1ffea938cd40f" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 44753 } } }
I20260807 10:56:11.573824 4069 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32edabcfe95f423cacb1ffea938cd40f [sys.catalog]: This master's current role is: LEADER
I20260807 10:56:11.576049 4077 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:56:11.582898 4077 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:56:11.584051 4053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:11.561384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55851 (local address 127.3.73.254:44753)
0807 10:56:11.561614 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:11.561625 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:11.562211 (+ 586us) server_negotiation.cc:407] Connection header received
0807 10:56:11.562422 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:11.562426 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:11.562509 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:11.562601 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:11.564576 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:11.565485 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:11.567460 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:11.568146 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:11.568295 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:11.573030 (+ 4735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:11.573042 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:11.573050 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:56:11.573114 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:11.578703 (+ 5589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:11.580166 (+ 1463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:11.580175 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:11.580181 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:11.580536 (+ 355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:11.581000 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:11.581007 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:11.581013 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:11.581593 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:11.583176 (+ 1583us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:11.583336 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 10:56:11.583594 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 10:56:11.597625 4077 catalog_manager.cc:1347] Generated new cluster ID: ad00a16021f543508f530c4e0fb2c13a
I20260807 10:56:11.597783 4077 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:56:11.612512 4077 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:56:11.614609 4077 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:56:11.626530 4077 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32edabcfe95f423cacb1ffea938cd40f: Generated new TSK 0
I20260807 10:56:11.627684 4077 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 10:56:11.641799 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 3964
I20260807 10:56:11.643162 4043 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:56:11.740115 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 3964
2026-08-07T10:56:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2562 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:56:11 dist-test-slave-x5fp krb5kdc[4102](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:56:11 dist-test-slave-x5fp krb5kdc[4102](info): set up 2 sockets
Aug 07 10:56:11 dist-test-slave-x5fp krb5kdc[4102](info): commencing operation
krb5kdc: starting...
W20260807 10:56:13.138862 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.353s user 0.000s sys 0.063s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:56:13 dist-test-slave-x5fp krb5kdc[4102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:56:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:56:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.73.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
I20260807 10:56:13.553870 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:41201
--webserver_interface=127.3.73.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:44123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:41201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:56:13.946640 4123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:56:13.946995 4123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:56:13.947094 4123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:56:13.955338 4123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:56:13.955538 4123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:56:13.955636 4123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:56:13.955742 4123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:56:13.955822 4123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:56:13.970762 4123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:44123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:41201
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:41201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:56:13.972304 4123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:56:13.973843 4123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:56:13.987677 4132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:56:13.987744 4134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:56:13.987679 4138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:56:13.988525 4123 server_base.cc:1034] running on GCE node
I20260807 10:56:13.989519 4123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:56:13.991101 4123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:56:13.992339 4123 hybrid_clock.cc:648] HybridClock initialized: now 1786100173992279 us; error 56 us; skew 500 ppm
I20260807 10:56:13.992741 4123 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:56:13 dist-test-slave-x5fp krb5kdc[4102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100173, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:56:13.996869 4123 init.cc:377] Logged in from keytab as kudu/127.3.73.254@KRBTEST.COM (short username kudu)
I20260807 10:56:13.998685 4123 webserver.cc:460] Webserver started at http://127.3.73.254:44261/ using document root <none> and password file <none>
I20260807 10:56:13.999464 4123 fs_manager.cc:359] Metadata directory not provided
I20260807 10:56:13.999606 4123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:56:13.999960 4123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:56:14.003757 4123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "61498b0957654c42b7e25fc97d78cc8e"
format_stamp: "Formatted at 2026-08-07 10:56:13 on dist-test-slave-x5fp"
server_key: "4160df52cd4d0ad96c679273392e38b8"
server_key_iv: "dc14a1532ae9940023b0296e1ffa2c34"
server_key_version: "encryptionkey@0"
I20260807 10:56:14.004777 4123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "61498b0957654c42b7e25fc97d78cc8e"
format_stamp: "Formatted at 2026-08-07 10:56:13 on dist-test-slave-x5fp"
server_key: "4160df52cd4d0ad96c679273392e38b8"
server_key_iv: "dc14a1532ae9940023b0296e1ffa2c34"
server_key_version: "encryptionkey@0"
I20260807 10:56:14.011070 4123 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 10:56:14.015816 4154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:56:14.017426 4123 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 10:56:14.017673 4123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "61498b0957654c42b7e25fc97d78cc8e"
format_stamp: "Formatted at 2026-08-07 10:56:13 on dist-test-slave-x5fp"
server_key: "4160df52cd4d0ad96c679273392e38b8"
server_key_iv: "dc14a1532ae9940023b0296e1ffa2c34"
server_key_version: "encryptionkey@0"
I20260807 10:56:14.017928 4123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:56:14.034088 4123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:56:14.039045 4123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:56:14.039395 4123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:56:14.060261 4123 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:41201
I20260807 10:56:14.060324 4227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:41201 every 8 connection(s)
I20260807 10:56:14.061869 4123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:56:14.066114 4229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:56:14.066946 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 4123
I20260807 10:56:14.067144 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:56:14.067759 3367 external_mini_cluster.cc:1428] Setting key 6b4af578e76720f3464db85913041292
I20260807 10:56:14.082365 4229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61498b0957654c42b7e25fc97d78cc8e: Bootstrap starting.
I20260807 10:56:14.084825 4233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:14.069781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37765 (local address 127.3.73.254:41201)
0807 10:56:14.070673 (+ 892us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:14.070690 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:14.070735 (+ 45us) server_negotiation.cc:407] Connection header received
0807 10:56:14.073473 (+ 2738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:14.073514 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:14.074195 (+ 681us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:14.074686 (+ 491us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:14.075396 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:14.077081 (+ 1685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:14.079212 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:14.080232 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:14.080432 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:14.081867 (+ 1435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:14.081876 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:14.081895 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:56:14.082639 (+ 744us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:14.082756 (+ 117us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:56:14.082789 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:14.082894 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:14.083150 (+ 256us) server_negotiation.cc:299] Negotiation successful
0807 10:56:14.083463 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":652,"thread_start_us":412,"threads_started":1}
I20260807 10:56:14.086305 4229 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61498b0957654c42b7e25fc97d78cc8e: Neither blocks nor log segments found. Creating new log.
I20260807 10:56:14.087754 4229 log.cc:824] T 00000000000000000000000000000000 P 61498b0957654c42b7e25fc97d78cc8e: Log is configured to *not* fsync() on all Append() calls
I20260807 10:56:14.089514 3367 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 10:56:14.091097 4229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61498b0957654c42b7e25fc97d78cc8e: No bootstrap required, opened a new log
I20260807 10:56:14.101157 4229 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61498b0957654c42b7e25fc97d78cc8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61498b0957654c42b7e25fc97d78cc8e" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41201 } }
I20260807 10:56:14.101718 4229 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61498b0957654c42b7e25fc97d78cc8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:56:14.101796 4233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:14.094342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39099 (local address 127.3.73.254:41201)
0807 10:56:14.094556 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:14.094563 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:14.094913 (+ 350us) server_negotiation.cc:407] Connection header received
0807 10:56:14.095087 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:14.095093 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:14.095182 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:14.095320 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:14.095880 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:14.097087 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:14.098724 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:14.099471 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:14.099613 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:14.100874 (+ 1261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:14.100880 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:14.100885 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:56:14.100912 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:14.100966 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:56:14.100987 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:14.101090 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:14.101348 (+ 258us) server_negotiation.cc:299] Negotiation successful
0807 10:56:14.101452 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 10:56:14.101869 4229 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61498b0957654c42b7e25fc97d78cc8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61498b0957654c42b7e25fc97d78cc8e, State: Initialized, Role: FOLLOWER
I20260807 10:56:14.102618 4229 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61498b0957654c42b7e25fc97d78cc8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61498b0957654c42b7e25fc97d78cc8e" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41201 } }
I20260807 10:56:14.102886 4229 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61498b0957654c42b7e25fc97d78cc8e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:56:14.103051 4229 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61498b0957654c42b7e25fc97d78cc8e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:56:14.103236 4229 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61498b0957654c42b7e25fc97d78cc8e [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:56:14.105721 4229 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61498b0957654c42b7e25fc97d78cc8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61498b0957654c42b7e25fc97d78cc8e" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41201 } }
I20260807 10:56:14.106278 4229 leader_election.cc:302] T 00000000000000000000000000000000 P 61498b0957654c42b7e25fc97d78cc8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61498b0957654c42b7e25fc97d78cc8e; no voters:
I20260807 10:56:14.107069 4229 leader_election.cc:288] T 00000000000000000000000000000000 P 61498b0957654c42b7e25fc97d78cc8e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:56:14.107283 4253 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61498b0957654c42b7e25fc97d78cc8e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:56:14.109372 4229 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61498b0957654c42b7e25fc97d78cc8e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:56:14.111605 4253 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61498b0957654c42b7e25fc97d78cc8e [term 1 LEADER]: Becoming Leader. State: Replica: 61498b0957654c42b7e25fc97d78cc8e, State: Running, Role: LEADER
I20260807 10:56:14.112514 4253 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61498b0957654c42b7e25fc97d78cc8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61498b0957654c42b7e25fc97d78cc8e" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41201 } }
I20260807 10:56:14.116204 4254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61498b0957654c42b7e25fc97d78cc8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "61498b0957654c42b7e25fc97d78cc8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61498b0957654c42b7e25fc97d78cc8e" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41201 } } }
I20260807 10:56:14.116765 4255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61498b0957654c42b7e25fc97d78cc8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61498b0957654c42b7e25fc97d78cc8e. Latest consensus state: current_term: 1 leader_uuid: "61498b0957654c42b7e25fc97d78cc8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61498b0957654c42b7e25fc97d78cc8e" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41201 } } }
I20260807 10:56:14.116945 4254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61498b0957654c42b7e25fc97d78cc8e [sys.catalog]: This master's current role is: LEADER
I20260807 10:56:14.117090 4255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61498b0957654c42b7e25fc97d78cc8e [sys.catalog]: This master's current role is: LEADER
W20260807 10:56:14.121465 4274 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 61498b0957654c42b7e25fc97d78cc8e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 10:56:14.121702 4274 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 10:56:14.121886 4275 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:56:14.124048 4275 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:56:14.132701 4275 catalog_manager.cc:1347] Generated new cluster ID: 9eae4ec5105845e1ab7a3583aa524bd0
I20260807 10:56:14.132880 4275 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:56:14.159409 4275 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:56:14.161478 4275 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:56:14.173805 4275 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 61498b0957654c42b7e25fc97d78cc8e: Generated new TSK 0
I20260807 10:56:14.175158 4275 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 10:56:14.190004 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 4123
W20260807 10:56:14.191233 4222 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37765
W20260807 10:56:14.192026 3367 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 4123: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 10:56:14.192193 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 4123
2026-08-07T10:56:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2456 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:56:14 dist-test-slave-x5fp krb5kdc[4282](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:56:14 dist-test-slave-x5fp krb5kdc[4282](info): set up 2 sockets
Aug 07 10:56:14 dist-test-slave-x5fp krb5kdc[4282](info): commencing operation
krb5kdc: starting...
W20260807 10:56:15.580329 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.338s user 0.000s sys 0.067s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:56:15 dist-test-slave-x5fp krb5kdc[4282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:56:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:56:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.73.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
I20260807 10:56:15.994848 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:41199
--webserver_interface=127.3.73.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:39837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:41199
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:56:16.378049 4298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:56:16.378434 4298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:56:16.378644 4298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:56:16.386686 4298 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:56:16.386845 4298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:56:16.386947 4298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:56:16.387040 4298 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:56:16.387127 4298 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:56:16.398561 4298 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:39837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:41199
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:41199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:56:16.400089 4298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:56:16.401691 4298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:56:16.415032 4305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:56:16.415210 4306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:56:16.415256 4309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:56:16.416105 4298 server_base.cc:1034] running on GCE node
I20260807 10:56:16.417141 4298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:56:16.418823 4298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:56:16.420086 4298 hybrid_clock.cc:648] HybridClock initialized: now 1786100176420031 us; error 54 us; skew 500 ppm
I20260807 10:56:16.420487 4298 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:56:16 dist-test-slave-x5fp krb5kdc[4282](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100176, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:56:16.424710 4298 init.cc:377] Logged in from keytab as kudu/127.3.73.254@KRBTEST.COM (short username kudu)
I20260807 10:56:16.426786 4298 webserver.cc:460] Webserver started at http://127.3.73.254:41915/ using document root <none> and password file <none>
I20260807 10:56:16.427676 4298 fs_manager.cc:359] Metadata directory not provided
I20260807 10:56:16.427856 4298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:56:16.428227 4298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:56:16.431988 4298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "e6132cec3df9401b9b313387d75bb3a4"
format_stamp: "Formatted at 2026-08-07 10:56:16 on dist-test-slave-x5fp"
server_key: "8a51145a78bf06465cdf57e53389d296"
server_key_iv: "fc7e260abe0f9b325bafaf4ec4ab5257"
server_key_version: "encryptionkey@0"
I20260807 10:56:16.433075 4298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "e6132cec3df9401b9b313387d75bb3a4"
format_stamp: "Formatted at 2026-08-07 10:56:16 on dist-test-slave-x5fp"
server_key: "8a51145a78bf06465cdf57e53389d296"
server_key_iv: "fc7e260abe0f9b325bafaf4ec4ab5257"
server_key_version: "encryptionkey@0"
I20260807 10:56:16.439605 4298 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 10:56:16.444048 4318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:56:16.445840 4298 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 10:56:16.446076 4298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "e6132cec3df9401b9b313387d75bb3a4"
format_stamp: "Formatted at 2026-08-07 10:56:16 on dist-test-slave-x5fp"
server_key: "8a51145a78bf06465cdf57e53389d296"
server_key_iv: "fc7e260abe0f9b325bafaf4ec4ab5257"
server_key_version: "encryptionkey@0"
I20260807 10:56:16.446336 4298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:56:16.453540 4298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:56:16.464672 4298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:56:16.465188 4298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:56:16.487649 4298 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:41199
I20260807 10:56:16.487689 4402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:41199 every 8 connection(s)
I20260807 10:56:16.489342 4298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:56:16.493500 4403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:56:16.496415 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 4298
I20260807 10:56:16.496623 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:56:16.497207 3367 external_mini_cluster.cc:1428] Setting key a07b3e7052952c6c76f57dcf19a3f8bc
I20260807 10:56:16.505766 4408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:16.499238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50709 (local address 127.3.73.254:41199)
0807 10:56:16.500133 (+ 895us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:16.500145 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:16.500191 (+ 46us) server_negotiation.cc:407] Connection header received
0807 10:56:16.502013 (+ 1822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:16.502040 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:16.502423 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:16.502781 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:16.503965 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:16.503971 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:16.503985 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:56:16.504004 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:16.504070 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:56:16.504094 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:16.504281 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:16.504415 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 10:56:16.504645 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":419,"threads_started":1}
I20260807 10:56:16.507153 4403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6132cec3df9401b9b313387d75bb3a4: Bootstrap starting.
I20260807 10:56:16.510144 3367 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 10:56:16.510942 4403 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6132cec3df9401b9b313387d75bb3a4: Neither blocks nor log segments found. Creating new log.
I20260807 10:56:16.512369 4403 log.cc:824] T 00000000000000000000000000000000 P e6132cec3df9401b9b313387d75bb3a4: Log is configured to *not* fsync() on all Append() calls
I20260807 10:56:16.516083 4403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6132cec3df9401b9b313387d75bb3a4: No bootstrap required, opened a new log
I20260807 10:56:16.518726 4408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:16.515997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44409 (local address 127.3.73.254:41199)
0807 10:56:16.516201 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:16.516208 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:16.516570 (+ 362us) server_negotiation.cc:407] Connection header received
0807 10:56:16.516713 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:16.516737 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:16.516815 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:16.516946 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:16.517921 (+ 975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:16.517929 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:16.517935 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 10:56:16.517955 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:16.518005 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 10:56:16.518025 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:16.518166 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:16.518335 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 10:56:16.518410 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 10:56:16.525334 4403 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6132cec3df9401b9b313387d75bb3a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6132cec3df9401b9b313387d75bb3a4" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41199 } }
I20260807 10:56:16.525930 4403 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6132cec3df9401b9b313387d75bb3a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 10:56:16.526052 4403 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6132cec3df9401b9b313387d75bb3a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6132cec3df9401b9b313387d75bb3a4, State: Initialized, Role: FOLLOWER
I20260807 10:56:16.526626 4403 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6132cec3df9401b9b313387d75bb3a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6132cec3df9401b9b313387d75bb3a4" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41199 } }
I20260807 10:56:16.526858 4403 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6132cec3df9401b9b313387d75bb3a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 10:56:16.526966 4403 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6132cec3df9401b9b313387d75bb3a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 10:56:16.527091 4403 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6132cec3df9401b9b313387d75bb3a4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:56:16.529516 4403 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6132cec3df9401b9b313387d75bb3a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6132cec3df9401b9b313387d75bb3a4" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41199 } }
I20260807 10:56:16.530042 4403 leader_election.cc:302] T 00000000000000000000000000000000 P e6132cec3df9401b9b313387d75bb3a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6132cec3df9401b9b313387d75bb3a4; no voters:
I20260807 10:56:16.530778 4403 leader_election.cc:288] T 00000000000000000000000000000000 P e6132cec3df9401b9b313387d75bb3a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 10:56:16.530936 4416 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6132cec3df9401b9b313387d75bb3a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:56:16.532052 4416 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6132cec3df9401b9b313387d75bb3a4 [term 1 LEADER]: Becoming Leader. State: Replica: e6132cec3df9401b9b313387d75bb3a4, State: Running, Role: LEADER
I20260807 10:56:16.532620 4416 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6132cec3df9401b9b313387d75bb3a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6132cec3df9401b9b313387d75bb3a4" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41199 } }
I20260807 10:56:16.533226 4403 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6132cec3df9401b9b313387d75bb3a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:56:16.535727 4419 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6132cec3df9401b9b313387d75bb3a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6132cec3df9401b9b313387d75bb3a4. Latest consensus state: current_term: 1 leader_uuid: "e6132cec3df9401b9b313387d75bb3a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6132cec3df9401b9b313387d75bb3a4" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41199 } } }
I20260807 10:56:16.535754 4418 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6132cec3df9401b9b313387d75bb3a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6132cec3df9401b9b313387d75bb3a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6132cec3df9401b9b313387d75bb3a4" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 41199 } } }
I20260807 10:56:16.536300 4419 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6132cec3df9401b9b313387d75bb3a4 [sys.catalog]: This master's current role is: LEADER
I20260807 10:56:16.536346 4418 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6132cec3df9401b9b313387d75bb3a4 [sys.catalog]: This master's current role is: LEADER
I20260807 10:56:16.537398 4426 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:56:16.543215 4426 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:56:16.553184 4426 catalog_manager.cc:1347] Generated new cluster ID: 7174e13e05fd481ea832dab6a490732d
I20260807 10:56:16.553335 4426 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:56:16.569470 4426 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:56:16.570086 4426 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:56:16.583873 4426 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6132cec3df9401b9b313387d75bb3a4: Generated new TSK 0
I20260807 10:56:16.585254 4426 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 10:56:16.591651 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 4298
W20260807 10:56:16.593011 4396 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:50709
W20260807 10:56:16.593745 3367 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 4298: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260807 10:56:16.593918 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 4298
2026-08-07T10:56:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2399 ms)
[----------] 18 tests from AuthTokenIssuingTest (44704 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:56:16 dist-test-slave-x5fp krb5kdc[4449](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:56:16 dist-test-slave-x5fp krb5kdc[4449](info): set up 2 sockets
Aug 07 10:56:16 dist-test-slave-x5fp krb5kdc[4449](info): commencing operation
krb5kdc: starting...
W20260807 10:56:18.006310 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.364s user 0.000s sys 0.066s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:56:18 dist-test-slave-x5fp krb5kdc[4449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:56:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:56:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.73.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
I20260807 10:56:18.429888 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:45421
--webserver_interface=127.3.73.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:35337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:45421,127.3.73.253:39835,127.3.73.252:46049
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:56:18.822687 4468 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 10:56:18.823136 4468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:56:18.823271 4468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:56:18.823410 4468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:56:18.830967 4468 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:56:18.831141 4468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:56:18.831259 4468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:56:18.831352 4468 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:56:18.831449 4468 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:56:18.842163 4468 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:45421,127.3.73.253:39835,127.3.73.252:46049
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:45421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:56:18.843685 4468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:56:18.845162 4468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:56:18.858116 4475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:56:18.858129 4479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:56:18.858116 4476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:56:18.858875 4468 server_base.cc:1034] running on GCE node
I20260807 10:56:18.859804 4468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:56:18.861426 4468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:56:18.862658 4468 hybrid_clock.cc:648] HybridClock initialized: now 1786100178862522 us; error 105 us; skew 500 ppm
I20260807 10:56:18.863023 4468 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:56:18 dist-test-slave-x5fp krb5kdc[4449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100178, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:56:18.866813 4468 init.cc:377] Logged in from keytab as kudu/127.3.73.254@KRBTEST.COM (short username kudu)
I20260807 10:56:18.869014 4468 webserver.cc:460] Webserver started at http://127.3.73.254:42255/ using document root <none> and password file <none>
I20260807 10:56:18.869817 4468 fs_manager.cc:359] Metadata directory not provided
I20260807 10:56:18.869920 4468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:56:18.870304 4468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:56:18.873669 4468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "77e55322b1fe4dbba2c6b2b52b9f995b"
format_stamp: "Formatted at 2026-08-07 10:56:18 on dist-test-slave-x5fp"
server_key: "ea46866d8ed0ef5ce887c4ccd1c3fcff"
server_key_iv: "ae68dc6ebc8ae62bb2687af47a3bb748"
server_key_version: "encryptionkey@0"
I20260807 10:56:18.874573 4468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "77e55322b1fe4dbba2c6b2b52b9f995b"
format_stamp: "Formatted at 2026-08-07 10:56:18 on dist-test-slave-x5fp"
server_key: "ea46866d8ed0ef5ce887c4ccd1c3fcff"
server_key_iv: "ae68dc6ebc8ae62bb2687af47a3bb748"
server_key_version: "encryptionkey@0"
I20260807 10:56:18.880575 4468 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260807 10:56:18.885141 4488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:56:18.887014 4468 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 10:56:18.887343 4468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "77e55322b1fe4dbba2c6b2b52b9f995b"
format_stamp: "Formatted at 2026-08-07 10:56:18 on dist-test-slave-x5fp"
server_key: "ea46866d8ed0ef5ce887c4ccd1c3fcff"
server_key_iv: "ae68dc6ebc8ae62bb2687af47a3bb748"
server_key_version: "encryptionkey@0"
I20260807 10:56:18.887622 4468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:56:18.903609 4468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:56:18.908604 4468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:56:18.908975 4468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:56:18.931707 4468 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:45421
I20260807 10:56:18.931744 4567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:45421 every 8 connection(s)
I20260807 10:56:18.933312 4468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:56:18.933682 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 4468
I20260807 10:56:18.933919 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:56:18.934814 3367 external_mini_cluster.cc:1428] Setting key c06cac47a4fac576c2adeee6fbe9d6d5
I20260807 10:56:18.937654 4568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:56:18.946105 4568 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45421 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 10:56:18 dist-test-slave-x5fp krb5kdc[4449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100178, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.73.253@KRBTEST.COM; defaulting to no policy
I20260807 10:56:18.975122 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:18.950425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:45421 (local address 127.0.0.1:53726)
0807 10:56:18.952870 (+ 2445us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:18.952923 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 10:56:18.954985 (+ 2062us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:56:18.955967 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:56:18.955973 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:56:18.956889 (+ 916us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:56:18.957271 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:56:18.957296 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:56:18.958957 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:56:18.958962 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:56:18.960099 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:56:18.960113 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:56:18.961191 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:56:18.961197 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:56:18.961373 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:18.962102 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:56:18.962128 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:56:18.964987 (+ 2859us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:56:18.970336 (+ 5349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:56:18.970349 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:56:18.970361 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:56:18.970794 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:56:18.971435 (+ 641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:56:18.971438 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:56:18.971441 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:56:18.971615 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:56:18.972882 (+ 1267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:56:18.972889 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:56:18.973117 (+ 228us) client_negotiation.cc:769] Sending connection context
0807 10:56:18.973361 (+ 244us) client_negotiation.cc:240] Negotiation successful
0807 10:56:18.973885 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2142,"thread_start_us":1942,"threads_started":1}
I20260807 10:56:18.975176 4573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:18.950425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53726 (local address 127.3.73.254:45421)
0807 10:56:18.952312 (+ 1887us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:18.952326 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:18.953063 (+ 737us) server_negotiation.cc:407] Connection header received
0807 10:56:18.955431 (+ 2368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:18.955438 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:18.955707 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:18.955936 (+ 229us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:18.957514 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:18.958778 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:18.960265 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:18.960933 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:18.961145 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:18.965334 (+ 4189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:18.965349 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:18.965372 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:56:18.965423 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:18.970002 (+ 4579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:18.971002 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:18.971015 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:18.971020 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:18.971265 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:18.971758 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:18.971767 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:18.971771 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:18.972674 (+ 903us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:18.972981 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:18.973608 (+ 627us) server_negotiation.cc:299] Negotiation successful
0807 10:56:18.974095 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":129,"server-negotiator.queue_time_us":1644,"thread_start_us":1359,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.253@KRBTEST.COM" created.
I20260807 10:56:18.978819 4568 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39835 } has no permanent_uuid. Determining permanent_uuid...
I20260807 10:56:18.980064 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:18.979583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:39835 (local address 127.0.0.1:56780)
0807 10:56:18.979818 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:18.979934 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.253:39835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260807 10:56:18.980427 4489 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.73.253:39835: connect: Connection refused (error 111)
I20260807 10:56:18.982343 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:18.981858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:39835 (local address 127.0.0.1:56792)
0807 10:56:18.982080 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:18.982166 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.253:39835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260807 10:56:18.982733 4568 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.253:39835: Network error: Client connection negotiation failed: client connection to 127.3.73.253:39835: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.253.keytab.
Entry for principal kudu/127.3.73.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.253.keytab.
I20260807 10:56:19.024824 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.253:39835
--webserver_interface=127.3.73.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.253.keytab
--principal=kudu/127.3.73.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:35337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:45421,127.3.73.253:39835,127.3.73.252:46049
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 10:56:19.032152 4568 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39835 } attempt: 1
I20260807 10:56:19.033499 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:19.033014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:39835 (local address 127.0.0.1:56808)
0807 10:56:19.033231 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:19.033334 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.253:39835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260807 10:56:19.034647 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:19.034251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:39835 (local address 127.0.0.1:56824)
0807 10:56:19.034418 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:19.034495 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.253:39835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260807 10:56:19.034883 4568 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.253:39835: Network error: Client connection negotiation failed: client connection to 127.3.73.253:39835: connect: Connection refused (error 111)
I20260807 10:56:19.103204 4568 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39835 } attempt: 2
I20260807 10:56:19.104619 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:19.103977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:39835 (local address 127.0.0.1:56834)
0807 10:56:19.104297 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:19.104423 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.253:39835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260807 10:56:19.105891 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:19.105419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:39835 (local address 127.0.0.1:56838)
0807 10:56:19.105586 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:19.105683 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.253:39835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 10:56:19.106132 4568 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.253:39835: Network error: Client connection negotiation failed: client connection to 127.3.73.253:39835: connect: Connection refused (error 111)
I20260807 10:56:19.197405 4568 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39835 } attempt: 3
I20260807 10:56:19.198827 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:19.198254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:39835 (local address 127.0.0.1:56848)
0807 10:56:19.198517 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:19.198635 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.253:39835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260807 10:56:19.200172 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:19.199636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:39835 (local address 127.0.0.1:56864)
0807 10:56:19.199848 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:19.199959 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.253:39835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260807 10:56:19.200428 4568 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.253:39835: Network error: Client connection negotiation failed: client connection to 127.3.73.253:39835: connect: Connection refused (error 111)
I20260807 10:56:19.343721 4568 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39835 } attempt: 4
I20260807 10:56:19.345316 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:19.344572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:39835 (local address 127.0.0.1:56874)
0807 10:56:19.344946 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:19.345121 (+ 175us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.253:39835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201}
I20260807 10:56:19.346629 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:19.346146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:39835 (local address 127.0.0.1:56888)
0807 10:56:19.346337 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:19.346452 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.253:39835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260807 10:56:19.346905 4568 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.253:39835: Network error: Client connection negotiation failed: client connection to 127.3.73.253:39835: connect: Connection refused (error 111)
W20260807 10:56:19.404701 4577 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 10:56:19.405200 4577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:56:19.405344 4577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:56:19.405454 4577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:56:19.413311 4577 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:56:19.413432 4577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:56:19.413532 4577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:56:19.413599 4577 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:56:19.413686 4577 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:56:19.424381 4577 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:45421,127.3.73.253:39835,127.3.73.252:46049
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.253.keytab
--principal=kudu/127.3.73.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.253:39835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.73.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:56:19.425906 4577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:56:19.427367 4577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:56:19.440706 4588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:56:19.440682 4590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:56:19.440629 4587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:56:19.441691 4577 server_base.cc:1034] running on GCE node
I20260807 10:56:19.442574 4577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:56:19.444141 4577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:56:19.445372 4577 hybrid_clock.cc:648] HybridClock initialized: now 1786100179445305 us; error 40 us; skew 500 ppm
I20260807 10:56:19.445742 4577 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:56:19 dist-test-slave-x5fp krb5kdc[4449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100179, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:56:19.449441 4577 init.cc:377] Logged in from keytab as kudu/127.3.73.253@KRBTEST.COM (short username kudu)
I20260807 10:56:19.451193 4577 webserver.cc:460] Webserver started at http://127.3.73.253:43003/ using document root <none> and password file <none>
I20260807 10:56:19.451982 4577 fs_manager.cc:359] Metadata directory not provided
I20260807 10:56:19.452122 4577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:56:19.452476 4577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:56:19.455834 4577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-1/data/instance:
uuid: "ca76da013221491ba90911d847663c34"
format_stamp: "Formatted at 2026-08-07 10:56:19 on dist-test-slave-x5fp"
server_key: "0ee350ff05cd54e2d266a5bf136e5856"
server_key_iv: "952baf56db488e419ff7a44653542215"
server_key_version: "encryptionkey@0"
I20260807 10:56:19.456710 4577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-1/wal/instance:
uuid: "ca76da013221491ba90911d847663c34"
format_stamp: "Formatted at 2026-08-07 10:56:19 on dist-test-slave-x5fp"
server_key: "0ee350ff05cd54e2d266a5bf136e5856"
server_key_iv: "952baf56db488e419ff7a44653542215"
server_key_version: "encryptionkey@0"
I20260807 10:56:19.462749 4577 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260807 10:56:19.466984 4600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:56:19.468583 4577 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 10:56:19.468842 4577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-1/wal
uuid: "ca76da013221491ba90911d847663c34"
format_stamp: "Formatted at 2026-08-07 10:56:19 on dist-test-slave-x5fp"
server_key: "0ee350ff05cd54e2d266a5bf136e5856"
server_key_iv: "952baf56db488e419ff7a44653542215"
server_key_version: "encryptionkey@0"
I20260807 10:56:19.469110 4577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:56:19.485539 4577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:56:19.490724 4577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:56:19.491075 4577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:56:19.512976 4577 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.253:39835
I20260807 10:56:19.513025 4680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.253:39835 every 8 connection(s)
I20260807 10:56:19.514565 4577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-1/data/info.pb
I20260807 10:56:19.516350 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 4577
I20260807 10:56:19.516556 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-1/wal/instance
I20260807 10:56:19.517172 3367 external_mini_cluster.cc:1428] Setting key 24c97ad52fe77ec8f84c8f953944727c
I20260807 10:56:19.519622 4681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:56:19.530444 4681 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45421 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 10:56:19 dist-test-slave-x5fp krb5kdc[4449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100179, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.253@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.73.252@KRBTEST.COM; defaulting to no policy
I20260807 10:56:19.559559 4684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:19.534296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53730 (local address 127.3.73.254:45421)
0807 10:56:19.534926 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:19.534935 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:19.536000 (+ 1065us) server_negotiation.cc:407] Connection header received
0807 10:56:19.539068 (+ 3068us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:19.539073 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:19.539170 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:19.539282 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:19.541982 (+ 2700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:19.542816 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:19.545878 (+ 3062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:19.546371 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:19.546497 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:19.551943 (+ 5446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:19.551950 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:19.551954 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:56:19.552008 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:19.555492 (+ 3484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:19.556563 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:19.556569 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:19.556572 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:19.556800 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:19.557489 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:19.557500 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:19.557503 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:19.557842 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:19.558045 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:19.558863 (+ 818us) server_negotiation.cc:299] Negotiation successful
0807 10:56:19.559134 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":300,"threads_started":1}
I20260807 10:56:19.560755 4686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:19.534893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:45421 (local address 127.0.0.1:53730)
0807 10:56:19.535778 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:19.535840 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 10:56:19.538658 (+ 2818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:56:19.539612 (+ 954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:56:19.539628 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:56:19.540629 (+ 1001us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:56:19.541622 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:56:19.541656 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:56:19.543029 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:56:19.543038 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:56:19.545566 (+ 2528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:56:19.545593 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:56:19.546714 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:56:19.546725 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:56:19.547124 (+ 399us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:19.548118 (+ 994us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:56:19.548159 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:56:19.551516 (+ 3357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:56:19.555730 (+ 4214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:56:19.555744 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:56:19.555771 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:56:19.556373 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:56:19.557048 (+ 675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:56:19.557054 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:56:19.557059 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:56:19.557335 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:56:19.558015 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:56:19.558024 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:56:19.558453 (+ 429us) client_negotiation.cc:769] Sending connection context
0807 10:56:19.558752 (+ 299us) client_negotiation.cc:240] Negotiation successful
0807 10:56:19.559387 (+ 635us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":378,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.252@KRBTEST.COM" created.
I20260807 10:56:19.562942 4681 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39835 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 10:56:19 dist-test-slave-x5fp krb5kdc[4449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100179, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.253@KRBTEST.COM for kudu/127.3.73.253@KRBTEST.COM
I20260807 10:56:19.581094 4686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:19.563714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:39835 (local address 127.0.0.1:56900)
0807 10:56:19.564091 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:19.564116 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 10:56:19.564246 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:56:19.565367 (+ 1121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:56:19.565375 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:56:19.565985 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:56:19.566219 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:56:19.566234 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:56:19.567796 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:56:19.567804 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:56:19.569337 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:56:19.569360 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:56:19.570409 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:56:19.570418 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:56:19.570647 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:19.571416 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:56:19.571447 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:56:19.574434 (+ 2987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:56:19.578274 (+ 3840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:56:19.578280 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:56:19.578285 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:56:19.578740 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:56:19.579176 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:56:19.579182 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:56:19.579186 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:56:19.579332 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:56:19.579966 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:56:19.579976 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:56:19.580152 (+ 176us) client_negotiation.cc:769] Sending connection context
0807 10:56:19.580300 (+ 148us) client_negotiation.cc:240] Negotiation successful
0807 10:56:19.580629 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192}
I20260807 10:56:19.581223 4689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:19.564085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56900 (local address 127.3.73.253:39835)
0807 10:56:19.564905 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:19.564911 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:19.564939 (+ 28us) server_negotiation.cc:407] Connection header received
0807 10:56:19.564990 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:19.564996 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:19.565187 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:19.565325 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:19.566437 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:19.567627 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:19.569542 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:19.570232 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:19.570353 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:19.574701 (+ 4348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:19.574715 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:19.574730 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:56:19.574766 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:19.578103 (+ 3337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:19.578914 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:19.578923 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:19.578925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:19.579057 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:19.579438 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:19.579445 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:19.579448 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:19.579836 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:19.580084 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:19.580467 (+ 383us) server_negotiation.cc:299] Negotiation successful
0807 10:56:19.580793 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":637,"thread_start_us":447,"threads_started":1}
I20260807 10:56:19.584458 4681 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.73.252" port: 46049 } has no permanent_uuid. Determining permanent_uuid...
I20260807 10:56:19.585974 4686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:19.585238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:46049 (local address 127.0.0.1:34856)
0807 10:56:19.585504 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:19.585694 (+ 190us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:46049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260807 10:56:19.586325 4608 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.73.252:46049: connect: Connection refused (error 111)
I20260807 10:56:19.588392 4686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:19.587832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:46049 (local address 127.0.0.1:34862)
0807 10:56:19.588072 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:19.588202 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:46049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260807 10:56:19.588686 4681 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.252:46049: Network error: Client connection negotiation failed: client connection to 127.3.73.252:46049: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.252.keytab.
Entry for principal kudu/127.3.73.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.252.keytab.
I20260807 10:56:19.616242 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.252:46049
--webserver_interface=127.3.73.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.252.keytab
--principal=kudu/127.3.73.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:35337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:45421,127.3.73.253:39835,127.3.73.252:46049
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 10:56:19.638152 4681 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.252" port: 46049 } attempt: 1
I20260807 10:56:19.639662 4686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:19.638996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:46049 (local address 127.0.0.1:34878)
0807 10:56:19.639260 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:19.639409 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:46049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260807 10:56:19.640970 4686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:19.640459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:46049 (local address 127.0.0.1:34882)
0807 10:56:19.640653 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:19.640797 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:46049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 10:56:19.641242 4681 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.252:46049: Network error: Client connection negotiation failed: client connection to 127.3.73.252:46049: connect: Connection refused (error 111)
I20260807 10:56:19.646225 4568 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39835 } attempt: 5
Aug 07 10:56:19 dist-test-slave-x5fp krb5kdc[4449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100178, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for kudu/127.3.73.253@KRBTEST.COM
I20260807 10:56:19.662813 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:19.647084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:39835 (local address 127.0.0.1:56914)
0807 10:56:19.647317 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:19.647341 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0807 10:56:19.647533 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:56:19.647942 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:56:19.647946 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:56:19.648383 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:56:19.648567 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:56:19.648578 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:56:19.650216 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:56:19.650221 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:56:19.651166 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:56:19.651180 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:56:19.652358 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:56:19.652363 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:56:19.652574 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:19.653404 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:56:19.653425 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:56:19.655558 (+ 2133us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:56:19.659868 (+ 4310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:56:19.659873 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:56:19.659877 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:56:19.660362 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:56:19.660861 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:56:19.660866 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:56:19.660869 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:56:19.661141 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:56:19.661899 (+ 758us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:56:19.661903 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:56:19.662095 (+ 192us) client_negotiation.cc:769] Sending connection context
0807 10:56:19.662252 (+ 157us) client_negotiation.cc:240] Negotiation successful
0807 10:56:19.662506 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260807 10:56:19.662973 4689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:19.647177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56914 (local address 127.3.73.253:39835)
0807 10:56:19.647355 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:19.647361 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:19.647484 (+ 123us) server_negotiation.cc:407] Connection header received
0807 10:56:19.647667 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:19.647676 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:19.647772 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:19.647897 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:19.648755 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:19.650029 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:19.651332 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:19.652154 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:19.652319 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:19.655820 (+ 3501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:19.655829 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:19.655837 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:56:19.655904 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:19.659672 (+ 3768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:19.660502 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:19.660509 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:19.660513 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:19.660682 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:19.661271 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:19.661277 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:19.661280 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:19.661674 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:19.661820 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:19.662392 (+ 572us) server_negotiation.cc:299] Negotiation successful
0807 10:56:19.662647 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 10:56:19.664251 4568 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.73.252" port: 46049 } has no permanent_uuid. Determining permanent_uuid...
I20260807 10:56:19.665663 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:19.665168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:46049 (local address 127.0.0.1:34888)
0807 10:56:19.665374 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:19.665473 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:46049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260807 10:56:19.667055 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:19.666626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:46049 (local address 127.0.0.1:34896)
0807 10:56:19.666825 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:19.666908 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:46049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260807 10:56:19.667376 4568 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.252:46049: Network error: Client connection negotiation failed: client connection to 127.3.73.252:46049: connect: Connection refused (error 111)
I20260807 10:56:19.709537 4681 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.252" port: 46049 } attempt: 2
I20260807 10:56:19.711089 4686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:19.710455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:46049 (local address 127.0.0.1:34912)
0807 10:56:19.710721 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:19.710839 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:46049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260807 10:56:19.712405 4686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:19.711961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:46049 (local address 127.0.0.1:34918)
0807 10:56:19.712139 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:19.712238 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:46049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 10:56:19.712779 4681 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.252:46049: Network error: Client connection negotiation failed: client connection to 127.3.73.252:46049: connect: Connection refused (error 111)
I20260807 10:56:19.718739 4568 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.252" port: 46049 } attempt: 1
I20260807 10:56:19.720314 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:19.719771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:46049 (local address 127.0.0.1:34934)
0807 10:56:19.720013 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:19.720116 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:46049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260807 10:56:19.721902 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:19.721345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:46049 (local address 127.0.0.1:34938)
0807 10:56:19.721589 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:19.721760 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:46049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260807 10:56:19.722167 4568 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.252:46049: Network error: Client connection negotiation failed: client connection to 127.3.73.252:46049: connect: Connection refused (error 111)
I20260807 10:56:19.790504 4568 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.252" port: 46049 } attempt: 2
I20260807 10:56:19.792510 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:19.791721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:46049 (local address 127.0.0.1:34952)
0807 10:56:19.792129 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:19.792242 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:46049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
I20260807 10:56:19.794143 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:19.793558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:46049 (local address 127.0.0.1:34954)
0807 10:56:19.793843 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:19.793950 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:46049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260807 10:56:19.794440 4568 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.252:46049: Network error: Client connection negotiation failed: client connection to 127.3.73.252:46049: connect: Connection refused (error 111)
I20260807 10:56:19.804078 4681 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.252" port: 46049 } attempt: 3
I20260807 10:56:19.806007 4686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:19.805185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:46049 (local address 127.0.0.1:34968)
0807 10:56:19.805570 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:19.805724 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:46049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197}
I20260807 10:56:19.807515 4686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:19.806954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:46049 (local address 127.0.0.1:34980)
0807 10:56:19.807235 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:19.807350 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:46049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147}
W20260807 10:56:19.807897 4681 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.252:46049: Network error: Client connection negotiation failed: client connection to 127.3.73.252:46049: connect: Connection refused (error 111)
I20260807 10:56:19.900699 4568 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.252" port: 46049 } attempt: 3
I20260807 10:56:19.902343 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:19.901695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:46049 (local address 127.0.0.1:34984)
0807 10:56:19.901969 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:19.902097 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:46049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260807 10:56:19.903668 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:19.903173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:46049 (local address 127.0.0.1:34986)
0807 10:56:19.903387 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:19.903516 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:46049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260807 10:56:19.903980 4568 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.252:46049: Network error: Client connection negotiation failed: client connection to 127.3.73.252:46049: connect: Connection refused (error 111)
I20260807 10:56:19.951242 4681 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.252" port: 46049 } attempt: 4
I20260807 10:56:19.952632 4686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:19.951955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:46049 (local address 127.0.0.1:34988)
0807 10:56:19.952255 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:19.952386 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:46049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260807 10:56:19.953879 4686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:19.953511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:46049 (local address 127.0.0.1:35002)
0807 10:56:19.953682 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:19.953759 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:46049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 10:56:19.954190 4681 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.252:46049: Network error: Client connection negotiation failed: client connection to 127.3.73.252:46049: connect: Connection refused (error 111)
W20260807 10:56:20.041863 4692 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 10:56:20.042266 4692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:56:20.042404 4692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:56:20.042550 4692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:56:20.050248 4692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:56:20.050364 4692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:56:20.050426 4692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:56:20.050472 4692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:56:20.050540 4692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:56:20.061404 4692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:35337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:45421,127.3.73.253:39835,127.3.73.252:46049
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.252.keytab
--principal=kudu/127.3.73.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.252:46049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.73.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:56:20.062776 4692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:56:20.064141 4692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:56:20.077950 4699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:56:20.077926 4704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:56:20.077980 4700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:56:20.078548 4692 server_base.cc:1034] running on GCE node
I20260807 10:56:20.079885 4692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:56:20.081236 4568 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.252" port: 46049 } attempt: 4
I20260807 10:56:20.081948 4692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:56:20.082621 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:20.082072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:46049 (local address 127.0.0.1:35012)
0807 10:56:20.082370 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:20.082478 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:46049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190}
I20260807 10:56:20.083250 4692 hybrid_clock.cc:648] HybridClock initialized: now 1786100180083142 us; error 83 us; skew 500 ppm
I20260807 10:56:20.083648 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:20.083293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:46049 (local address 127.0.0.1:35026)
0807 10:56:20.083443 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:20.083525 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:46049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 10:56:20.083666 4692 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
W20260807 10:56:20.083890 4568 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.252:46049: Network error: Client connection negotiation failed: client connection to 127.3.73.252:46049: connect: Connection refused (error 111)
Aug 07 10:56:20 dist-test-slave-x5fp krb5kdc[4449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100180, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:56:20.087816 4692 init.cc:377] Logged in from keytab as kudu/127.3.73.252@KRBTEST.COM (short username kudu)
I20260807 10:56:20.089610 4692 webserver.cc:460] Webserver started at http://127.3.73.252:40009/ using document root <none> and password file <none>
I20260807 10:56:20.090392 4692 fs_manager.cc:359] Metadata directory not provided
I20260807 10:56:20.090494 4692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:56:20.090823 4692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:56:20.094393 4692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-2/data/instance:
uuid: "c27efe5bfcf34624968c7ce80b402a37"
format_stamp: "Formatted at 2026-08-07 10:56:20 on dist-test-slave-x5fp"
server_key: "37ed82f3f2a80bbe1b79ed34df073098"
server_key_iv: "26093a9df4782c9ea545f7725837c62f"
server_key_version: "encryptionkey@0"
I20260807 10:56:20.095268 4692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-2/wal/instance:
uuid: "c27efe5bfcf34624968c7ce80b402a37"
format_stamp: "Formatted at 2026-08-07 10:56:20 on dist-test-slave-x5fp"
server_key: "37ed82f3f2a80bbe1b79ed34df073098"
server_key_iv: "26093a9df4782c9ea545f7725837c62f"
server_key_version: "encryptionkey@0"
I20260807 10:56:20.101451 4692 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260807 10:56:20.105834 4713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:56:20.107539 4692 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260807 10:56:20.107726 4692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-2/wal
uuid: "c27efe5bfcf34624968c7ce80b402a37"
format_stamp: "Formatted at 2026-08-07 10:56:20 on dist-test-slave-x5fp"
server_key: "37ed82f3f2a80bbe1b79ed34df073098"
server_key_iv: "26093a9df4782c9ea545f7725837c62f"
server_key_version: "encryptionkey@0"
I20260807 10:56:20.107976 4692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:56:20.124264 4692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:56:20.129648 4692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:56:20.129928 4692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:56:20.152072 4692 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.252:46049
I20260807 10:56:20.152108 4792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.252:46049 every 8 connection(s)
I20260807 10:56:20.153723 4692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-2/data/info.pb
I20260807 10:56:20.158061 4793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:56:20.160394 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 4692
I20260807 10:56:20.160640 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786099767749326-3367-0/minicluster-data/master-2/wal/instance
I20260807 10:56:20.161177 3367 external_mini_cluster.cc:1428] Setting key 1dc7a8d9d88221943153c71ef52d1ab2
I20260807 10:56:20.166347 4793 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45421 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 10:56:20 dist-test-slave-x5fp krb5kdc[4449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.252@KRBTEST.COM
Aug 07 10:56:20 dist-test-slave-x5fp krb5kdc[4449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100180, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.252@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:56:20.188102 4798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:20.169247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53736 (local address 127.3.73.254:45421)
0807 10:56:20.169908 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:20.169917 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:20.170397 (+ 480us) server_negotiation.cc:407] Connection header received
0807 10:56:20.170662 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:20.170667 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:20.170749 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:20.170874 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:20.172783 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:20.173627 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:20.175737 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:20.176495 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:20.176675 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:20.181119 (+ 4444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:20.181127 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:20.181134 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:56:20.181184 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:20.184440 (+ 3256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:20.185633 (+ 1193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:20.185639 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:20.185642 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:20.185884 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:20.186306 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:20.186309 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:20.186311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:20.186577 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:20.186771 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:20.187508 (+ 737us) server_negotiation.cc:299] Negotiation successful
0807 10:56:20.187752 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":301,"threads_started":1}
I20260807 10:56:20.189750 4801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:20.169207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:45421 (local address 127.0.0.1:53736)
0807 10:56:20.170246 (+ 1039us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:20.170295 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 10:56:20.170494 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:56:20.170955 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:56:20.170963 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:56:20.171980 (+ 1017us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:56:20.172360 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:56:20.172376 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:56:20.173811 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:56:20.173817 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:56:20.175521 (+ 1704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:56:20.175539 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:56:20.176856 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:56:20.176864 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:56:20.177009 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:20.177719 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:56:20.177745 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:56:20.180770 (+ 3025us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:56:20.184617 (+ 3847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:56:20.184628 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:56:20.184640 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:56:20.185487 (+ 847us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:56:20.186015 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:56:20.186020 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:56:20.186023 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:56:20.186184 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:56:20.186709 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:56:20.186719 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:56:20.187009 (+ 290us) client_negotiation.cc:769] Sending connection context
0807 10:56:20.187352 (+ 343us) client_negotiation.cc:240] Negotiation successful
0807 10:56:20.187999 (+ 647us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":803,"mutex_wait_us":51,"thread_start_us":603,"threads_started":1}
I20260807 10:56:20.189774 4797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:20.163441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35040 (local address 127.3.73.252:46049)
0807 10:56:20.164556 (+ 1115us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:20.164571 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:20.164629 (+ 58us) server_negotiation.cc:407] Connection header received
0807 10:56:20.166583 (+ 1954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:20.166609 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:20.167085 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:20.167522 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:20.169349 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:20.171184 (+ 1835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:20.173245 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:20.174344 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:20.174571 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:20.180089 (+ 5518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:20.180100 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:20.180121 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:56:20.180166 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:20.183817 (+ 3651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:20.184968 (+ 1151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:20.184979 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:20.184982 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:20.185234 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:20.185758 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:20.185763 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:20.185765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:20.186572 (+ 807us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:20.186959 (+ 387us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:20.187286 (+ 327us) server_negotiation.cc:299] Negotiation successful
0807 10:56:20.188134 (+ 848us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":148,"server-negotiator.queue_time_us":858,"thread_start_us":557,"threads_started":1}
I20260807 10:56:20.191686 4793 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39835 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 10:56:20 dist-test-slave-x5fp krb5kdc[4449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100180, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.252@KRBTEST.COM for kudu/127.3.73.253@KRBTEST.COM
Aug 07 10:56:20 dist-test-slave-x5fp krb5kdc[4449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:56:20.209702 4801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:20.192602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:39835 (local address 127.0.0.1:56920)
0807 10:56:20.192926 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:20.192943 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 10:56:20.193091 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:56:20.193817 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:56:20.193821 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:56:20.194371 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:56:20.194529 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:56:20.194538 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:56:20.196268 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:56:20.196275 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:56:20.197503 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:56:20.197516 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:56:20.198554 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:56:20.198558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:56:20.198809 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:20.199548 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:56:20.199575 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:56:20.202269 (+ 2694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:56:20.206808 (+ 4539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:56:20.206813 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:56:20.206816 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:56:20.207283 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:56:20.207731 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:56:20.207736 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:56:20.207740 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:56:20.207902 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:56:20.208497 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:56:20.208502 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:56:20.208698 (+ 196us) client_negotiation.cc:769] Sending connection context
0807 10:56:20.208922 (+ 224us) client_negotiation.cc:240] Negotiation successful
0807 10:56:20.209280 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211}
I20260807 10:56:20.209751 4802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:20.192779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56920 (local address 127.3.73.253:39835)
0807 10:56:20.193429 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:20.193434 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:20.193452 (+ 18us) server_negotiation.cc:407] Connection header received
0807 10:56:20.193553 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:20.193560 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:20.193645 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:20.193840 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:20.194743 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:20.196108 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:20.197669 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:20.198379 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:20.198533 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:20.202535 (+ 4002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:20.202541 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:20.202545 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:56:20.202586 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:20.206469 (+ 3883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:20.207440 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:20.207445 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:20.207448 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:20.207596 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:20.208034 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:20.208038 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:20.208041 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:20.208369 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:20.208541 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:20.209043 (+ 502us) server_negotiation.cc:299] Negotiation successful
0807 10:56:20.209289 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":330,"threads_started":1}
I20260807 10:56:20.210911 4793 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.73.252" port: 46049 } has no permanent_uuid. Determining permanent_uuid...
I20260807 10:56:20.211181 4798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:20.193528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53746 (local address 127.3.73.254:45421)
0807 10:56:20.193691 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:20.193696 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:20.193739 (+ 43us) server_negotiation.cc:407] Connection header received
0807 10:56:20.193947 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:20.193952 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:20.194050 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:20.194156 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:20.195908 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:20.196987 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:20.198648 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:20.199296 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:20.199434 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:20.204323 (+ 4889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:20.204328 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:20.204332 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:56:20.204370 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:20.207519 (+ 3149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:20.208396 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:20.208402 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:20.208410 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:20.208567 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:20.209035 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:20.209040 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:20.209042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:20.209340 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:20.209664 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:20.210632 (+ 968us) server_negotiation.cc:299] Negotiation successful
0807 10:56:20.210913 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
Aug 07 10:56:20 dist-test-slave-x5fp krb5kdc[4449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100180, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.252@KRBTEST.COM for kudu/127.3.73.252@KRBTEST.COM
Aug 07 10:56:20 dist-test-slave-x5fp krb5kdc[4449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.253@KRBTEST.COM
I20260807 10:56:20.226642 4797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:20.211914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35056 (local address 127.3.73.252:46049)
0807 10:56:20.212082 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:20.212087 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:20.212102 (+ 15us) server_negotiation.cc:407] Connection header received
0807 10:56:20.212166 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:20.212171 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:20.212237 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:20.212337 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:20.213298 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:20.214380 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:20.216085 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:20.216507 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:20.216653 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:20.220499 (+ 3846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:20.220505 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:20.220513 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:56:20.220545 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:20.223765 (+ 3220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:20.224465 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:20.224473 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:20.224476 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:20.224614 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:20.225058 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:20.225063 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:20.225066 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:20.225359 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:20.225515 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:20.225894 (+ 379us) server_negotiation.cc:299] Negotiation successful
0807 10:56:20.226335 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":188,"server-negotiator.queue_time_us":52}
I20260807 10:56:20.226686 4801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:20.211653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:46049 (local address 127.0.0.1:35056)
0807 10:56:20.211877 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:20.211897 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 10:56:20.212021 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:56:20.212369 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:56:20.212376 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:56:20.212925 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:56:20.213144 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:56:20.213157 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:56:20.214569 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:56:20.214573 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:56:20.215946 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:56:20.215960 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:56:20.216678 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:56:20.216684 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:56:20.216932 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:20.217639 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:56:20.217665 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:56:20.220296 (+ 2631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:56:20.223915 (+ 3619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:56:20.223921 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:56:20.223925 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:56:20.224339 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:56:20.224782 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:56:20.224787 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:56:20.224791 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:56:20.224920 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:56:20.225475 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:56:20.225480 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:56:20.225624 (+ 144us) client_negotiation.cc:769] Sending connection context
0807 10:56:20.225811 (+ 187us) client_negotiation.cc:240] Negotiation successful
0807 10:56:20.226185 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"mutex_wait_us":3}
I20260807 10:56:20.230971 4802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:20.213413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56924 (local address 127.3.73.253:39835)
0807 10:56:20.213604 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:20.213609 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:20.213648 (+ 39us) server_negotiation.cc:407] Connection header received
0807 10:56:20.213845 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:20.213848 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:20.213912 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:20.213987 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:20.215767 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:20.217043 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:20.218897 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:20.219330 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:20.219491 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:20.224582 (+ 5091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:20.224588 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:20.224591 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:56:20.224630 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:20.227883 (+ 3253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:20.228647 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:20.228653 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:20.228656 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:20.228874 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:20.229304 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:20.229310 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:20.229316 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:20.229719 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:20.230006 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:20.230306 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 10:56:20.230579 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 10:56:20.234157 4793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c27efe5bfcf34624968c7ce80b402a37: Bootstrap starting.
I20260807 10:56:20.237066 3367 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 10:56:20.238346 4793 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c27efe5bfcf34624968c7ce80b402a37: Neither blocks nor log segments found. Creating new log.
I20260807 10:56:20.239579 4793 log.cc:824] T 00000000000000000000000000000000 P c27efe5bfcf34624968c7ce80b402a37: Log is configured to *not* fsync() on all Append() calls
I20260807 10:56:20.243187 4793 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c27efe5bfcf34624968c7ce80b402a37: No bootstrap required, opened a new log
I20260807 10:56:20.240060 4532 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "77e55322b1fe4dbba2c6b2b52b9f995b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53746
I20260807 10:56:20.253463 4681 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.252" port: 46049 } attempt: 5
I20260807 10:56:20.254923 4793 raft_consensus.cc:348] T 00000000000000000000000000000000 P c27efe5bfcf34624968c7ce80b402a37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e55322b1fe4dbba2c6b2b52b9f995b" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45421 } } peers { permanent_uuid: "ca76da013221491ba90911d847663c34" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39835 } } peers { permanent_uuid: "c27efe5bfcf34624968c7ce80b402a37" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 46049 } }
I20260807 10:56:20.255435 4793 raft_consensus.cc:729] T 00000000000000000000000000000000 P c27efe5bfcf34624968c7ce80b402a37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c27efe5bfcf34624968c7ce80b402a37, State: Initialized, Role: FOLLOWER
I20260807 10:56:20.255936 4793 consensus_queue.cc:260] T 00000000000000000000000000000000 P c27efe5bfcf34624968c7ce80b402a37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e55322b1fe4dbba2c6b2b52b9f995b" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45421 } } peers { permanent_uuid: "ca76da013221491ba90911d847663c34" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39835 } } peers { permanent_uuid: "c27efe5bfcf34624968c7ce80b402a37" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 46049 } }
I20260807 10:56:20.257154 4804 sys_catalog.cc:455] T 00000000000000000000000000000000 P c27efe5bfcf34624968c7ce80b402a37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e55322b1fe4dbba2c6b2b52b9f995b" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45421 } } peers { permanent_uuid: "ca76da013221491ba90911d847663c34" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39835 } } peers { permanent_uuid: "c27efe5bfcf34624968c7ce80b402a37" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 46049 } } }
I20260807 10:56:20.257606 4804 sys_catalog.cc:458] T 00000000000000000000000000000000 P c27efe5bfcf34624968c7ce80b402a37 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 10:56:20.258615 4793 sys_catalog.cc:564] T 00000000000000000000000000000000 P c27efe5bfcf34624968c7ce80b402a37 [sys.catalog]: configured and running, proceeding with master startup.
Aug 07 10:56:20 dist-test-slave-x5fp krb5kdc[4449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100179, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.253@KRBTEST.COM for kudu/127.3.73.252@KRBTEST.COM
W20260807 10:56:20.275727 4820 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c27efe5bfcf34624968c7ce80b402a37: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 10:56:20.275950 4820 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 10:56:20.276041 4686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:20.254276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:46049 (local address 127.0.0.1:35060)
0807 10:56:20.254517 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:20.254539 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 10:56:20.254680 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:56:20.255135 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:56:20.255138 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:56:20.255683 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:56:20.255928 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:56:20.255940 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:56:20.257627 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:56:20.257633 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:56:20.259017 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:56:20.259036 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:56:20.260207 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:56:20.260213 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:56:20.260478 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:20.261267 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:56:20.261292 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:56:20.265140 (+ 3848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:56:20.271953 (+ 6813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:56:20.271960 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:56:20.271965 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:56:20.272634 (+ 669us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:56:20.273455 (+ 821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:56:20.273460 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:56:20.273464 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:56:20.273693 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:56:20.274767 (+ 1074us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:56:20.274775 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:56:20.274979 (+ 204us) client_negotiation.cc:769] Sending connection context
0807 10:56:20.275163 (+ 184us) client_negotiation.cc:240] Negotiation successful
0807 10:56:20.275562 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260807 10:56:20.277310 4797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:20.254407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35060 (local address 127.3.73.252:46049)
0807 10:56:20.254589 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:20.254596 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:20.254619 (+ 23us) server_negotiation.cc:407] Connection header received
0807 10:56:20.254857 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:20.254864 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:20.254966 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:20.255083 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:20.256108 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:20.257397 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:20.259202 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:20.260036 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:20.260180 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:20.265443 (+ 5263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:20.265453 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:20.265460 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:56:20.265524 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:20.271750 (+ 6226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:20.272874 (+ 1124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:20.272881 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:20.272886 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:20.273208 (+ 322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:20.273874 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:20.273883 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:20.273886 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:20.274475 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:20.274680 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:20.276438 (+ 1758us) server_negotiation.cc:299] Negotiation successful
0807 10:56:20.276819 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 10:56:20.284224 4681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca76da013221491ba90911d847663c34: Bootstrap starting.
I20260807 10:56:20.273083 4692 master_runner.cc:186] Error getting master registration for 127.3.73.254:45421: OK, instance_id { permanent_uuid: "77e55322b1fe4dbba2c6b2b52b9f995b" instance_seqno: 1786100178925856 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 10:56:20.285423 4692 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 10:56:20.289253 4681 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca76da013221491ba90911d847663c34: Neither blocks nor log segments found. Creating new log.
I20260807 10:56:20.290848 4681 log.cc:824] T 00000000000000000000000000000000 P ca76da013221491ba90911d847663c34: Log is configured to *not* fsync() on all Append() calls
I20260807 10:56:20.295078 4681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca76da013221491ba90911d847663c34: No bootstrap required, opened a new log
I20260807 10:56:20.308224 4681 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca76da013221491ba90911d847663c34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e55322b1fe4dbba2c6b2b52b9f995b" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45421 } } peers { permanent_uuid: "ca76da013221491ba90911d847663c34" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39835 } } peers { permanent_uuid: "c27efe5bfcf34624968c7ce80b402a37" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 46049 } }
I20260807 10:56:20.308813 4681 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca76da013221491ba90911d847663c34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca76da013221491ba90911d847663c34, State: Initialized, Role: FOLLOWER
I20260807 10:56:20.309504 4681 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca76da013221491ba90911d847663c34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e55322b1fe4dbba2c6b2b52b9f995b" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45421 } } peers { permanent_uuid: "ca76da013221491ba90911d847663c34" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39835 } } peers { permanent_uuid: "c27efe5bfcf34624968c7ce80b402a37" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 46049 } }
I20260807 10:56:20.310729 4829 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca76da013221491ba90911d847663c34 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e55322b1fe4dbba2c6b2b52b9f995b" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45421 } } peers { permanent_uuid: "ca76da013221491ba90911d847663c34" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39835 } } peers { permanent_uuid: "c27efe5bfcf34624968c7ce80b402a37" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 46049 } } }
I20260807 10:56:20.311187 4829 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca76da013221491ba90911d847663c34 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 10:56:20.312451 4681 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca76da013221491ba90911d847663c34 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 10:56:20.324399 4848 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ca76da013221491ba90911d847663c34: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 10:56:20.324633 4848 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 10:56:20.321588 4577 master_runner.cc:186] Error getting master registration for 127.3.73.254:45421: OK, instance_id { permanent_uuid: "77e55322b1fe4dbba2c6b2b52b9f995b" instance_seqno: 1786100178925856 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 10:56:20.333775 4577 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 10:56:20.361122 4568 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.252" port: 46049 } attempt: 5
Aug 07 10:56:20 dist-test-slave-x5fp krb5kdc[4449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100178, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for kudu/127.3.73.252@KRBTEST.COM
W20260807 10:56:20.382239 4720 tablet.cc:2369] T 00000000000000000000000000000000 P c27efe5bfcf34624968c7ce80b402a37: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:56:20.384862 4572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:20.361932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:46049 (local address 127.0.0.1:42746)
0807 10:56:20.362417 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:20.362442 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 10:56:20.362583 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:56:20.363059 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:56:20.363064 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:56:20.363692 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:56:20.363940 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:56:20.363958 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:56:20.365708 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:56:20.365715 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:56:20.367375 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:56:20.367426 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:56:20.368981 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:56:20.368990 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:56:20.369223 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:20.370036 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:56:20.370063 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:56:20.374198 (+ 4135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:56:20.379531 (+ 5333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:56:20.379539 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:56:20.379545 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:56:20.380229 (+ 684us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:56:20.381019 (+ 790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:56:20.381026 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:56:20.381030 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:56:20.381306 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:56:20.383526 (+ 2220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:56:20.383532 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:56:20.383772 (+ 240us) client_negotiation.cc:769] Sending connection context
0807 10:56:20.383970 (+ 198us) client_negotiation.cc:240] Negotiation successful
0807 10:56:20.384351 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314}
I20260807 10:56:20.385130 4797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:20.362392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42746 (local address 127.3.73.252:46049)
0807 10:56:20.362613 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:20.362622 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:20.362658 (+ 36us) server_negotiation.cc:407] Connection header received
0807 10:56:20.362735 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:20.362741 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:20.362859 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:20.362998 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:20.364139 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:20.365499 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:20.367624 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:20.368505 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:20.368903 (+ 398us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:20.374577 (+ 5674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:20.374586 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:20.374592 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:56:20.374646 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:20.379294 (+ 4648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:20.380447 (+ 1153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:20.380454 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:20.380458 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:20.380847 (+ 389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:20.381465 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:20.381471 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:20.381475 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:20.381884 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:20.383610 (+ 1726us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:20.384143 (+ 533us) server_negotiation.cc:299] Negotiation successful
0807 10:56:20.384511 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 10:56:20.394054 4568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77e55322b1fe4dbba2c6b2b52b9f995b: Bootstrap starting.
I20260807 10:56:20.399119 4568 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77e55322b1fe4dbba2c6b2b52b9f995b: Neither blocks nor log segments found. Creating new log.
I20260807 10:56:20.401407 4568 log.cc:824] T 00000000000000000000000000000000 P 77e55322b1fe4dbba2c6b2b52b9f995b: Log is configured to *not* fsync() on all Append() calls
I20260807 10:56:20.405865 4568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77e55322b1fe4dbba2c6b2b52b9f995b: No bootstrap required, opened a new log
I20260807 10:56:20.415167 4568 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77e55322b1fe4dbba2c6b2b52b9f995b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e55322b1fe4dbba2c6b2b52b9f995b" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45421 } } peers { permanent_uuid: "ca76da013221491ba90911d847663c34" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39835 } } peers { permanent_uuid: "c27efe5bfcf34624968c7ce80b402a37" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 46049 } }
I20260807 10:56:20.415863 4568 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77e55322b1fe4dbba2c6b2b52b9f995b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77e55322b1fe4dbba2c6b2b52b9f995b, State: Initialized, Role: FOLLOWER
I20260807 10:56:20.416647 4568 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77e55322b1fe4dbba2c6b2b52b9f995b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e55322b1fe4dbba2c6b2b52b9f995b" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45421 } } peers { permanent_uuid: "ca76da013221491ba90911d847663c34" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39835 } } peers { permanent_uuid: "c27efe5bfcf34624968c7ce80b402a37" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 46049 } }
I20260807 10:56:20.418467 4855 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77e55322b1fe4dbba2c6b2b52b9f995b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e55322b1fe4dbba2c6b2b52b9f995b" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45421 } } peers { permanent_uuid: "ca76da013221491ba90911d847663c34" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39835 } } peers { permanent_uuid: "c27efe5bfcf34624968c7ce80b402a37" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 46049 } } }
I20260807 10:56:20.419317 4855 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77e55322b1fe4dbba2c6b2b52b9f995b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 10:56:20.419765 4568 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77e55322b1fe4dbba2c6b2b52b9f995b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:56:20.419993 4532 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77e55322b1fe4dbba2c6b2b52b9f995b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 10:56:20.420209 4532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77e55322b1fe4dbba2c6b2b52b9f995b [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:56:20.424379 4532 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77e55322b1fe4dbba2c6b2b52b9f995b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e55322b1fe4dbba2c6b2b52b9f995b" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45421 } } peers { permanent_uuid: "ca76da013221491ba90911d847663c34" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39835 } } peers { permanent_uuid: "c27efe5bfcf34624968c7ce80b402a37" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 46049 } }
I20260807 10:56:20.427508 4643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "77e55322b1fe4dbba2c6b2b52b9f995b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ca76da013221491ba90911d847663c34"
I20260807 10:56:20.428153 4643 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca76da013221491ba90911d847663c34 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:56:20.432495 4643 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ca76da013221491ba90911d847663c34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77e55322b1fe4dbba2c6b2b52b9f995b in term 1.
I20260807 10:56:20.433552 4489 leader_election.cc:302] T 00000000000000000000000000000000 P 77e55322b1fe4dbba2c6b2b52b9f995b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77e55322b1fe4dbba2c6b2b52b9f995b, ca76da013221491ba90911d847663c34; no voters:
I20260807 10:56:20.435451 4752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "77e55322b1fe4dbba2c6b2b52b9f995b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c27efe5bfcf34624968c7ce80b402a37"
I20260807 10:56:20.436033 4752 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c27efe5bfcf34624968c7ce80b402a37 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:56:20.439880 4752 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c27efe5bfcf34624968c7ce80b402a37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 77e55322b1fe4dbba2c6b2b52b9f995b in term 1.
I20260807 10:56:20.440927 4532 leader_election.cc:288] T 00000000000000000000000000000000 P 77e55322b1fe4dbba2c6b2b52b9f995b [CANDIDATE]: Term 1 election: Requested vote from peers ca76da013221491ba90911d847663c34 (127.3.73.253:39835), c27efe5bfcf34624968c7ce80b402a37 (127.3.73.252:46049)
I20260807 10:56:20.448361 4855 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77e55322b1fe4dbba2c6b2b52b9f995b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:56:20.458460 4855 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77e55322b1fe4dbba2c6b2b52b9f995b [term 1 LEADER]: Becoming Leader. State: Replica: 77e55322b1fe4dbba2c6b2b52b9f995b, State: Running, Role: LEADER
I20260807 10:56:20.460556 4855 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77e55322b1fe4dbba2c6b2b52b9f995b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e55322b1fe4dbba2c6b2b52b9f995b" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45421 } } peers { permanent_uuid: "ca76da013221491ba90911d847663c34" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39835 } } peers { permanent_uuid: "c27efe5bfcf34624968c7ce80b402a37" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 46049 } }
I20260807 10:56:20.473912 4865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77e55322b1fe4dbba2c6b2b52b9f995b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77e55322b1fe4dbba2c6b2b52b9f995b. Latest consensus state: current_term: 1 leader_uuid: "77e55322b1fe4dbba2c6b2b52b9f995b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e55322b1fe4dbba2c6b2b52b9f995b" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45421 } } peers { permanent_uuid: "ca76da013221491ba90911d847663c34" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39835 } } peers { permanent_uuid: "c27efe5bfcf34624968c7ce80b402a37" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 46049 } } }
I20260807 10:56:20.474625 4865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77e55322b1fe4dbba2c6b2b52b9f995b [sys.catalog]: This master's current role is: LEADER
I20260807 10:56:20.480221 4875 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260807 10:56:20.494417 4611 tablet.cc:2369] T 00000000000000000000000000000000 P ca76da013221491ba90911d847663c34: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:56:20.511509 4885 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 10:56:20.504659 4875 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:56:20.555563 4752 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c27efe5bfcf34624968c7ce80b402a37 [term 1 FOLLOWER]: Refusing update from remote peer 77e55322b1fe4dbba2c6b2b52b9f995b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:56:20.557112 4643 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ca76da013221491ba90911d847663c34 [term 1 FOLLOWER]: Refusing update from remote peer 77e55322b1fe4dbba2c6b2b52b9f995b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:56:20.558401 4855 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 77e55322b1fe4dbba2c6b2b52b9f995b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca76da013221491ba90911d847663c34" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:56:20.560341 4855 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 77e55322b1fe4dbba2c6b2b52b9f995b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c27efe5bfcf34624968c7ce80b402a37" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 46049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:56:20.575120 4798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:20.487307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54530 (local address 127.3.73.254:45421)
0807 10:56:20.487519 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:20.487526 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:20.505964 (+ 18438us) server_negotiation.cc:407] Connection header received
0807 10:56:20.508857 (+ 2893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:20.508865 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:20.508967 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:20.509054 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:20.518078 (+ 9024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:20.519262 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:20.534924 (+ 15662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:20.535680 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:20.535804 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:20.557203 (+ 21399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:20.557212 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:20.557218 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:56:20.557270 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:20.571370 (+ 14100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:20.572310 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:20.572317 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:20.572321 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:20.572522 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:20.573097 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:20.573104 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:20.573107 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:20.573510 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:20.573810 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:20.574431 (+ 621us) server_negotiation.cc:299] Negotiation successful
0807 10:56:20.574712 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 10:56:20.579039 4797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:20.491952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42760 (local address 127.3.73.252:46049)
0807 10:56:20.492250 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:20.492258 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:20.507384 (+ 15126us) server_negotiation.cc:407] Connection header received
0807 10:56:20.511923 (+ 4539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:20.511930 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:20.512023 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:20.512131 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:20.521561 (+ 9430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:20.522752 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:20.537144 (+ 14392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:20.537898 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:20.538024 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:20.552537 (+ 14513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:20.552546 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:20.552551 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:56:20.552606 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:20.556866 (+ 4260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:20.567404 (+ 10538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:20.567411 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:20.567415 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:20.567605 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:20.568506 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:20.568513 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:20.568517 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:20.569063 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:20.569374 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:20.578294 (+ 8920us) server_negotiation.cc:299] Negotiation successful
0807 10:56:20.578607 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260807 10:56:20.582263 4802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:20.508050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34866 (local address 127.3.73.253:39835)
0807 10:56:20.508241 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:20.508248 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:20.513471 (+ 5223us) server_negotiation.cc:407] Connection header received
0807 10:56:20.514856 (+ 1385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:20.514862 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:20.514970 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:20.515102 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:20.530052 (+ 14950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:20.531426 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:20.541708 (+ 10282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:20.542471 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:20.542596 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:20.561225 (+ 18629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:20.561233 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:20.561238 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:56:20.561301 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:20.565308 (+ 4007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:20.568136 (+ 2828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:20.568143 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:20.568147 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:20.568328 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:20.570225 (+ 1897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:20.570232 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:20.570236 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:20.570810 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:20.571132 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:20.581516 (+ 10384us) server_negotiation.cc:299] Negotiation successful
0807 10:56:20.581830 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 10:56:20.595032 4829 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca76da013221491ba90911d847663c34 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77e55322b1fe4dbba2c6b2b52b9f995b. Latest consensus state: current_term: 1 leader_uuid: "77e55322b1fe4dbba2c6b2b52b9f995b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e55322b1fe4dbba2c6b2b52b9f995b" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45421 } } peers { permanent_uuid: "ca76da013221491ba90911d847663c34" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39835 } } peers { permanent_uuid: "c27efe5bfcf34624968c7ce80b402a37" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 46049 } } }
I20260807 10:56:20.595464 4829 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca76da013221491ba90911d847663c34 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 10:56:20.595492 4804 sys_catalog.cc:455] T 00000000000000000000000000000000 P c27efe5bfcf34624968c7ce80b402a37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77e55322b1fe4dbba2c6b2b52b9f995b. Latest consensus state: current_term: 1 leader_uuid: "77e55322b1fe4dbba2c6b2b52b9f995b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e55322b1fe4dbba2c6b2b52b9f995b" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45421 } } peers { permanent_uuid: "ca76da013221491ba90911d847663c34" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39835 } } peers { permanent_uuid: "c27efe5bfcf34624968c7ce80b402a37" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 46049 } } }
I20260807 10:56:20.595876 4804 sys_catalog.cc:458] T 00000000000000000000000000000000 P c27efe5bfcf34624968c7ce80b402a37 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 10:56:20.602154 4889 mvcc.cc:205] Tried to move back new op lower bound from 7315866339538251776 to 7315866339185647616. Current Snapshot: MvccSnapshot[applied={T|T < 7315866339538251776}]
I20260807 10:56:20.602309 4875 catalog_manager.cc:1347] Generated new cluster ID: 10596126151849188de493d6d87b112a
I20260807 10:56:20.602411 4875 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:56:20.604241 4855 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77e55322b1fe4dbba2c6b2b52b9f995b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "77e55322b1fe4dbba2c6b2b52b9f995b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e55322b1fe4dbba2c6b2b52b9f995b" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45421 } } peers { permanent_uuid: "ca76da013221491ba90911d847663c34" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39835 } } peers { permanent_uuid: "c27efe5bfcf34624968c7ce80b402a37" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 46049 } } }
I20260807 10:56:20.604585 4855 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77e55322b1fe4dbba2c6b2b52b9f995b [sys.catalog]: This master's current role is: LEADER
I20260807 10:56:20.606009 4855 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77e55322b1fe4dbba2c6b2b52b9f995b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "77e55322b1fe4dbba2c6b2b52b9f995b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e55322b1fe4dbba2c6b2b52b9f995b" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45421 } } peers { permanent_uuid: "ca76da013221491ba90911d847663c34" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39835 } } peers { permanent_uuid: "c27efe5bfcf34624968c7ce80b402a37" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 46049 } } }
I20260807 10:56:20.606269 4855 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77e55322b1fe4dbba2c6b2b52b9f995b [sys.catalog]: This master's current role is: LEADER
I20260807 10:56:20.617199 4829 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca76da013221491ba90911d847663c34 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "77e55322b1fe4dbba2c6b2b52b9f995b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e55322b1fe4dbba2c6b2b52b9f995b" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45421 } } peers { permanent_uuid: "ca76da013221491ba90911d847663c34" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39835 } } peers { permanent_uuid: "c27efe5bfcf34624968c7ce80b402a37" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 46049 } } }
I20260807 10:56:20.617528 4829 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca76da013221491ba90911d847663c34 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 10:56:20.627655 4875 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:56:20.629669 4875 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:56:20.632455 4804 sys_catalog.cc:455] T 00000000000000000000000000000000 P c27efe5bfcf34624968c7ce80b402a37 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "77e55322b1fe4dbba2c6b2b52b9f995b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "77e55322b1fe4dbba2c6b2b52b9f995b" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 45421 } } peers { permanent_uuid: "ca76da013221491ba90911d847663c34" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39835 } } peers { permanent_uuid: "c27efe5bfcf34624968c7ce80b402a37" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 46049 } } }
I20260807 10:56:20.632799 4804 sys_catalog.cc:458] T 00000000000000000000000000000000 P c27efe5bfcf34624968c7ce80b402a37 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 10:56:20.652794 4875 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77e55322b1fe4dbba2c6b2b52b9f995b: Generated new TSK 0
I20260807 10:56:20.654104 4875 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 10:56:20.775537 3367 mini_kdc.cc:348] Time spent kdestroy: real 0.105s user 0.001s sys 0.037s
W20260807 10:56:20.830772 4798 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:56:20.813088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54542 (local address 127.3.73.254:45421)
0807 10:56:20.813387 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:20.813400 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:20.825138 (+ 11738us) server_negotiation.cc:407] Connection header received
0807 10:56:20.828672 (+ 3534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:20.828680 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:20.828834 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:20.828938 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:20.830440 (+ 1502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54542: BlockingRecv error: recv got EOF from 127.0.0.1:54542 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260807 10:56:20.834262 4797 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:56:20.821320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42762 (local address 127.3.73.252:46049)
0807 10:56:20.821595 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:20.821607 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:20.831898 (+ 10291us) server_negotiation.cc:407] Connection header received
0807 10:56:20.832252 (+ 354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:20.832262 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:20.832373 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:20.832485 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:20.833936 (+ 1451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42762: BlockingRecv error: recv got EOF from 127.0.0.1:42762 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260807 10:56:20.834715 4802 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:56:20.825855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34878 (local address 127.3.73.253:39835)
0807 10:56:20.826123 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:20.826130 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:20.832097 (+ 5967us) server_negotiation.cc:407] Connection header received
0807 10:56:20.832410 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:20.832435 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:20.832534 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:20.832611 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:20.834429 (+ 1818us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34878: BlockingRecv error: recv got EOF from 127.0.0.1:34878 (error 108)
Metrics: {"server-negotiator.queue_time_us":126}
W20260807 10:56:20.886349 4802 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:56:20.873140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34886 (local address 127.3.73.253:39835)
0807 10:56:20.873421 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:20.873428 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:20.883358 (+ 9930us) server_negotiation.cc:407] Connection header received
0807 10:56:20.884019 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:20.884027 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:20.884219 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:20.884313 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:20.886048 (+ 1735us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34886: BlockingRecv error: recv got EOF from 127.0.0.1:34886 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
W20260807 10:56:20.935566 4802 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:56:20.923013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34890 (local address 127.3.73.253:39835)
0807 10:56:20.923234 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:20.923242 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:20.933465 (+ 10223us) server_negotiation.cc:407] Connection header received
0807 10:56:20.933871 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:20.933878 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:20.933985 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:20.934080 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:20.935300 (+ 1220us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34890: BlockingRecv error: recv got EOF from 127.0.0.1:34890 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260807 10:56:20.967494 4802 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:56:20.964555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34892 (local address 127.3.73.253:39835)
0807 10:56:20.964823 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:20.964831 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:20.965365 (+ 534us) server_negotiation.cc:407] Connection header received
0807 10:56:20.965592 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:20.965598 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:20.965700 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:20.965840 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:20.967211 (+ 1371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34892: BlockingRecv error: recv got EOF from 127.0.0.1:34892 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260807 10:56:20.978813 4802 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:56:20.976380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34898 (local address 127.3.73.253:39835)
0807 10:56:20.976667 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:20.976674 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:20.977229 (+ 555us) server_negotiation.cc:407] Connection header received
0807 10:56:20.977429 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:20.977435 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:20.977531 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:20.977637 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:20.978537 (+ 900us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34898: BlockingRecv error: recv got EOF from 127.0.0.1:34898 (error 108)
Metrics: {"server-negotiator.queue_time_us":117}
W20260807 10:56:20.993542 4802 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:56:20.990768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34914 (local address 127.3.73.253:39835)
0807 10:56:20.990971 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:20.990977 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:20.991565 (+ 588us) server_negotiation.cc:407] Connection header received
0807 10:56:20.991881 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:20.991888 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:20.992054 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:20.992185 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:20.993244 (+ 1059us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34914: BlockingRecv error: recv got EOF from 127.0.0.1:34914 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260807 10:56:21.017190 4802 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:56:21.014496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34918 (local address 127.3.73.253:39835)
0807 10:56:21.014673 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:21.014679 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:21.015295 (+ 616us) server_negotiation.cc:407] Connection header received
0807 10:56:21.015502 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:21.015509 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:21.015614 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:21.015736 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:21.016922 (+ 1186us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34918: BlockingRecv error: recv got EOF from 127.0.0.1:34918 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260807 10:56:21.055135 4802 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:56:21.052922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34932 (local address 127.3.73.253:39835)
0807 10:56:21.053180 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:21.053188 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:21.053508 (+ 320us) server_negotiation.cc:407] Connection header received
0807 10:56:21.053673 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:21.053679 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:21.053851 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:21.054001 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:21.054832 (+ 831us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34932: BlockingRecv error: recv got EOF from 127.0.0.1:34932 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
W20260807 10:56:21.125417 4802 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:56:21.123225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34936 (local address 127.3.73.253:39835)
0807 10:56:21.123455 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:21.123462 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:21.123824 (+ 362us) server_negotiation.cc:407] Connection header received
0807 10:56:21.124008 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:21.124015 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:21.124131 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:21.124269 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:21.125145 (+ 876us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34936: BlockingRecv error: recv got EOF from 127.0.0.1:34936 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260807 10:56:21.261162 4802 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:56:21.258902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34940 (local address 127.3.73.253:39835)
0807 10:56:21.259142 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:21.259150 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:21.259384 (+ 234us) server_negotiation.cc:407] Connection header received
0807 10:56:21.259578 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:21.259583 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:21.259694 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:21.259848 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:21.260920 (+ 1072us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34940: BlockingRecv error: recv got EOF from 127.0.0.1:34940 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
I20260807 10:56:21.278208 4820 catalog_manager.cc:1259] Loaded cluster ID: 10596126151849188de493d6d87b112a
I20260807 10:56:21.278330 4820 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c27efe5bfcf34624968c7ce80b402a37: loading cluster ID for follower catalog manager: success
I20260807 10:56:21.281085 4820 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c27efe5bfcf34624968c7ce80b402a37: acquiring CA information for follower catalog manager: success
I20260807 10:56:21.283273 4820 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c27efe5bfcf34624968c7ce80b402a37: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 10:56:21.326884 4848 catalog_manager.cc:1259] Loaded cluster ID: 10596126151849188de493d6d87b112a
I20260807 10:56:21.327013 4848 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ca76da013221491ba90911d847663c34: loading cluster ID for follower catalog manager: success
I20260807 10:56:21.329718 4848 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ca76da013221491ba90911d847663c34: acquiring CA information for follower catalog manager: success
I20260807 10:56:21.331838 4848 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ca76da013221491ba90911d847663c34: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 10:56:21.529747 4802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:21.522063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34942 (local address 127.3.73.253:39835)
0807 10:56:21.522351 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:21.522358 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:21.522679 (+ 321us) server_negotiation.cc:407] Connection header received
0807 10:56:21.522951 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:21.522956 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:21.523326 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:21.523497 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:56:21.524082 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:21.524981 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:21.527129 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:21.527620 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:21.527733 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:21.528358 (+ 625us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:56:21.529085 (+ 727us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:56:21.529207 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:21.529360 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 10:56:21.529437 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260807 10:56:21.542938 4997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:21.536518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42766 (local address 127.3.73.252:46049)
0807 10:56:21.537188 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:21.537195 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:21.537217 (+ 22us) server_negotiation.cc:407] Connection header received
0807 10:56:21.537451 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:21.537456 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:21.537622 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:21.537814 (+ 192us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:56:21.538240 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:21.539049 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:21.540464 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:21.540939 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:21.541085 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:21.541648 (+ 563us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:56:21.542201 (+ 553us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:56:21.542375 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:21.542520 (+ 145us) server_negotiation.cc:299] Negotiation successful
0807 10:56:21.542615 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":406,"threads_started":1}
I20260807 10:56:21.559834 4802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:21.551141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34952 (local address 127.3.73.253:39835)
0807 10:56:21.551380 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:21.551388 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:21.553593 (+ 2205us) server_negotiation.cc:407] Connection header received
0807 10:56:21.553929 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:21.553936 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:21.554037 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:21.554144 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:56:21.554874 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:21.555889 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:21.557869 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:21.558310 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:21.558492 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:21.558922 (+ 430us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:56:21.559074 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:56:21.559173 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:21.559355 (+ 182us) server_negotiation.cc:299] Negotiation successful
0807 10:56:21.559451 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 10:56:21.560719 4997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:21.550552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42778 (local address 127.3.73.252:46049)
0807 10:56:21.550758 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:21.550765 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:21.553489 (+ 2724us) server_negotiation.cc:407] Connection header received
0807 10:56:21.553680 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:21.553685 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:21.553760 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:21.553866 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:56:21.554547 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:21.555371 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:21.557990 (+ 2619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:21.558781 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:21.558907 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:21.559695 (+ 788us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:56:21.559852 (+ 157us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:56:21.559967 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:21.560313 (+ 346us) server_negotiation.cc:299] Negotiation successful
0807 10:56:21.560402 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 10:56:21.561347 5005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:21.550348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54554 (local address 127.3.73.254:45421)
0807 10:56:21.552264 (+ 1916us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:21.552274 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:21.552300 (+ 26us) server_negotiation.cc:407] Connection header received
0807 10:56:21.552474 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:21.552483 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:21.552813 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:21.554296 (+ 1483us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:56:21.554506 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:21.555557 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:21.558017 (+ 2460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:21.558757 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:21.558984 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:21.559678 (+ 694us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:56:21.560211 (+ 533us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:56:21.560902 (+ 691us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:21.560957 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 10:56:21.561058 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1734,"thread_start_us":1605,"threads_started":1}
I20260807 10:56:21.572180 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 4468
I20260807 10:56:21.573632 4563 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:56:21.690070 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 4468
W20260807 10:56:21.726890 4717 connection.cc:535] server connection from 127.0.0.1:42746 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 10:56:21.727571 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 4577
I20260807 10:56:21.728960 4671 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:56:21.841185 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 4577
I20260807 10:56:21.873426 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 4692
I20260807 10:56:21.874655 4788 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:56:21.995018 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 4692
2026-08-07T10:56:22Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5405 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 10:56:22 dist-test-slave-x5fp krb5kdc[5016](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 10:56:22 dist-test-slave-x5fp krb5kdc[5016](info): set up 2 sockets
Aug 07 10:56:22 dist-test-slave-x5fp krb5kdc[5016](info): commencing operation
krb5kdc: starting...
W20260807 10:56:23.387928 3367 mini_kdc.cc:121] Time spent starting KDC: real 1.340s user 0.000s sys 0.064s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/krb5kdc/test-user.keytab.
Aug 07 10:56:23 dist-test-slave-x5fp krb5kdc[5016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T10:56:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T10:56:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.73.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
Entry for principal kudu/127.3.73.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab.
I20260807 10:56:23.805171 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:37305
--webserver_interface=127.3.73.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:34325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:37305,127.3.73.253:39573,127.3.73.252:42165
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 10:56:24.200343 5038 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 10:56:24.200834 5038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:56:24.200984 5038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:56:24.201135 5038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:56:24.209076 5038 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:56:24.209220 5038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:56:24.209352 5038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:56:24.209442 5038 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:56:24.209558 5038 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:56:24.220616 5038 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:34325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:37305,127.3.73.253:39573,127.3.73.252:42165
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.254.keytab
--principal=kudu/127.3.73.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.254:37305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.73.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:56:24.222234 5038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:56:24.223742 5038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:56:24.237319 5047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:56:24.237604 5050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:56:24.238366 5048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:56:24.238956 5038 server_base.cc:1034] running on GCE node
I20260807 10:56:24.240062 5038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:56:24.241864 5038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:56:24.243172 5038 hybrid_clock.cc:648] HybridClock initialized: now 1786100184243103 us; error 36 us; skew 500 ppm
I20260807 10:56:24.243594 5038 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:56:24 dist-test-slave-x5fp krb5kdc[5016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100184, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:56:24.247299 5038 init.cc:377] Logged in from keytab as kudu/127.3.73.254@KRBTEST.COM (short username kudu)
I20260807 10:56:24.249296 5038 webserver.cc:460] Webserver started at http://127.3.73.254:43529/ using document root <none> and password file <none>
I20260807 10:56:24.250155 5038 fs_manager.cc:359] Metadata directory not provided
I20260807 10:56:24.250319 5038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:56:24.250731 5038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:56:24.254406 5038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-0/data/instance:
uuid: "a048b4545549414a9f6b3757cd1416c9"
format_stamp: "Formatted at 2026-08-07 10:56:24 on dist-test-slave-x5fp"
server_key: "07e63bdd82e4f8c3d5c7326f6f40e55c"
server_key_iv: "517d049618df03bb7a54fbe9b607213c"
server_key_version: "encryptionkey@0"
I20260807 10:56:24.255354 5038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-0/wal/instance:
uuid: "a048b4545549414a9f6b3757cd1416c9"
format_stamp: "Formatted at 2026-08-07 10:56:24 on dist-test-slave-x5fp"
server_key: "07e63bdd82e4f8c3d5c7326f6f40e55c"
server_key_iv: "517d049618df03bb7a54fbe9b607213c"
server_key_version: "encryptionkey@0"
I20260807 10:56:24.261641 5038 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260807 10:56:24.265645 5063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:56:24.267163 5038 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 10:56:24.267373 5038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-0/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-0/wal
uuid: "a048b4545549414a9f6b3757cd1416c9"
format_stamp: "Formatted at 2026-08-07 10:56:24 on dist-test-slave-x5fp"
server_key: "07e63bdd82e4f8c3d5c7326f6f40e55c"
server_key_iv: "517d049618df03bb7a54fbe9b607213c"
server_key_version: "encryptionkey@0"
I20260807 10:56:24.267638 5038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:56:24.290088 5038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:56:24.295370 5038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:56:24.295697 5038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:56:24.316694 5038 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.254:37305
I20260807 10:56:24.316865 5131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.254:37305 every 8 connection(s)
I20260807 10:56:24.318240 5038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-0/data/info.pb
I20260807 10:56:24.322527 5132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:56:24.327162 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 5038
I20260807 10:56:24.327374 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-0/wal/instance
I20260807 10:56:24.327841 3367 external_mini_cluster.cc:1428] Setting key 2dcc11f7a8ced2e9ffed1845456acf76
I20260807 10:56:24.330327 5132 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.73.254" port: 37305 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 10:56:24 dist-test-slave-x5fp krb5kdc[5016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100184, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:56:24.354574 5135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:24.333915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:37305 (local address 127.0.0.1:44630)
0807 10:56:24.334887 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:24.334943 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 10:56:24.337192 (+ 2249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:56:24.338108 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:56:24.338114 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:56:24.338879 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:56:24.339233 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:56:24.339251 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:56:24.340949 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:56:24.340953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:56:24.342067 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:56:24.342081 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:56:24.343070 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:56:24.343078 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:56:24.343255 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:24.344013 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:56:24.344037 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:56:24.346466 (+ 2429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:56:24.349994 (+ 3528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:56:24.350008 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:56:24.350023 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:56:24.350442 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:56:24.350888 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:56:24.350893 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:56:24.350896 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:56:24.351068 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:56:24.351913 (+ 845us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:56:24.351922 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:56:24.352161 (+ 239us) client_negotiation.cc:769] Sending connection context
0807 10:56:24.352446 (+ 285us) client_negotiation.cc:240] Negotiation successful
0807 10:56:24.353095 (+ 649us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"mutex_wait_us":134,"thread_start_us":424,"threads_started":1}
I20260807 10:56:24.354611 5136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:24.333915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44630 (local address 127.3.73.254:37305)
0807 10:56:24.335794 (+ 1879us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:24.335803 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:24.335855 (+ 52us) server_negotiation.cc:407] Connection header received
0807 10:56:24.337609 (+ 1754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:24.337615 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:24.337866 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:24.338095 (+ 229us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:24.339474 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:24.340706 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:24.342221 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:24.342813 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:24.343044 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:24.346719 (+ 3675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:24.346729 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:24.346744 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:56:24.346780 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:24.349827 (+ 3047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:24.350597 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:24.350604 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:24.350607 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:24.350763 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:24.351180 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:24.351187 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:24.351190 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:24.351755 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:24.351986 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:24.352575 (+ 589us) server_negotiation.cc:299] Negotiation successful
0807 10:56:24.352968 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1641,"thread_start_us":1393,"threads_started":1}
I20260807 10:56:24.358168 5132 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39573 } has no permanent_uuid. Determining permanent_uuid...
I20260807 10:56:24.359335 5135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:24.358924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:39573 (local address 127.0.0.1:38482)
0807 10:56:24.359097 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:24.359201 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.253:39573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260807 10:56:24.359603 5064 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.73.253:39573: connect: Connection refused (error 111)
I20260807 10:56:24.361363 5135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:24.360976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:39573 (local address 127.0.0.1:38484)
0807 10:56:24.361142 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:24.361215 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.253:39573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 10:56:24.361601 5132 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.253:39573: Network error: Client connection negotiation failed: client connection to 127.3.73.253:39573: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.3.73.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.253.keytab.
Entry for principal kudu/127.3.73.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.253.keytab.
I20260807 10:56:24.411046 5132 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39573 } attempt: 1
I20260807 10:56:24.412528 5135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:24.411927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:39573 (local address 127.0.0.1:38492)
0807 10:56:24.412157 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:24.412309 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.253:39573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260807 10:56:24.412642 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.253:39573
--webserver_interface=127.3.73.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.253.keytab
--principal=kudu/127.3.73.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:34325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:37305,127.3.73.253:39573,127.3.73.252:42165
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 10:56:24.413828 5135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:24.413370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:39573 (local address 127.0.0.1:38496)
0807 10:56:24.413563 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:24.413660 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.253:39573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260807 10:56:24.414065 5132 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.253:39573: Network error: Client connection negotiation failed: client connection to 127.3.73.253:39573: connect: Connection refused (error 111)
I20260807 10:56:24.482394 5132 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39573 } attempt: 2
I20260807 10:56:24.483711 5135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:24.483186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:39573 (local address 127.0.0.1:38512)
0807 10:56:24.483471 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:24.483563 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.253:39573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260807 10:56:24.484887 5135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:24.484441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:39573 (local address 127.0.0.1:38524)
0807 10:56:24.484595 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:24.484685 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.253:39573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 10:56:24.485081 5132 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.253:39573: Network error: Client connection negotiation failed: client connection to 127.3.73.253:39573: connect: Connection refused (error 111)
I20260807 10:56:24.576367 5132 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39573 } attempt: 3
I20260807 10:56:24.577646 5135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:24.577167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:39573 (local address 127.0.0.1:38536)
0807 10:56:24.577404 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:24.577495 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.253:39573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260807 10:56:24.579090 5135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:24.578621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:39573 (local address 127.0.0.1:38540)
0807 10:56:24.578827 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:24.578938 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.253:39573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260807 10:56:24.579384 5132 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.253:39573: Network error: Client connection negotiation failed: client connection to 127.3.73.253:39573: connect: Connection refused (error 111)
I20260807 10:56:24.722707 5132 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39573 } attempt: 4
I20260807 10:56:24.724316 5135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:24.723610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:39573 (local address 127.0.0.1:38546)
0807 10:56:24.724023 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:24.724130 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.253:39573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224}
I20260807 10:56:24.725637 5135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:24.725194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:39573 (local address 127.0.0.1:38550)
0807 10:56:24.725374 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:24.725481 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.253:39573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 10:56:24.725883 5132 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.253:39573: Network error: Client connection negotiation failed: client connection to 127.3.73.253:39573: connect: Connection refused (error 111)
W20260807 10:56:24.800463 5145 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 10:56:24.800964 5145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:56:24.801131 5145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:56:24.801275 5145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:56:24.809914 5145 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:56:24.810035 5145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:56:24.810143 5145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:56:24.810240 5145 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:56:24.810366 5145 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:56:24.821354 5145 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:34325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:37305,127.3.73.253:39573,127.3.73.252:42165
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.253.keytab
--principal=kudu/127.3.73.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.253:39573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.73.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:56:24.822834 5145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:56:24.824378 5145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:56:24.837566 5151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:56:24.837613 5155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:56:24.837635 5153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:56:24.838631 5145 server_base.cc:1034] running on GCE node
I20260807 10:56:24.839596 5145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:56:24.841297 5145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:56:24.842583 5145 hybrid_clock.cc:648] HybridClock initialized: now 1786100184842494 us; error 65 us; skew 500 ppm
I20260807 10:56:24.842967 5145 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 07 10:56:24 dist-test-slave-x5fp krb5kdc[5016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100184, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:56:24.846462 5145 init.cc:377] Logged in from keytab as kudu/127.3.73.253@KRBTEST.COM (short username kudu)
I20260807 10:56:24.848330 5145 webserver.cc:460] Webserver started at http://127.3.73.253:36951/ using document root <none> and password file <none>
I20260807 10:56:24.849222 5145 fs_manager.cc:359] Metadata directory not provided
I20260807 10:56:24.849371 5145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:56:24.849730 5145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:56:24.853127 5145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-1/data/instance:
uuid: "6f22e95eb2cd421caa9325952c2edda2"
format_stamp: "Formatted at 2026-08-07 10:56:24 on dist-test-slave-x5fp"
server_key: "6fe89b9fd8617e3bfffea3eaeb517338"
server_key_iv: "69738a128efe0b4d075177033ebee4ae"
server_key_version: "encryptionkey@0"
I20260807 10:56:24.854013 5145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-1/wal/instance:
uuid: "6f22e95eb2cd421caa9325952c2edda2"
format_stamp: "Formatted at 2026-08-07 10:56:24 on dist-test-slave-x5fp"
server_key: "6fe89b9fd8617e3bfffea3eaeb517338"
server_key_iv: "69738a128efe0b4d075177033ebee4ae"
server_key_version: "encryptionkey@0"
I20260807 10:56:24.859725 5145 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260807 10:56:24.863579 5166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:56:24.865149 5145 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 10:56:24.865362 5145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-1/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-1/wal
uuid: "6f22e95eb2cd421caa9325952c2edda2"
format_stamp: "Formatted at 2026-08-07 10:56:24 on dist-test-slave-x5fp"
server_key: "6fe89b9fd8617e3bfffea3eaeb517338"
server_key_iv: "69738a128efe0b4d075177033ebee4ae"
server_key_version: "encryptionkey@0"
I20260807 10:56:24.865617 5145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:56:24.891461 5145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:56:24.897204 5145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:56:24.897527 5145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:56:24.918776 5145 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.253:39573
I20260807 10:56:24.918808 5243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.253:39573 every 8 connection(s)
I20260807 10:56:24.920331 5145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-1/data/info.pb
I20260807 10:56:24.924355 5244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:56:24.926751 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 5145
I20260807 10:56:24.927026 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-1/wal/instance
I20260807 10:56:24.927568 3367 external_mini_cluster.cc:1428] Setting key 45c2b1b5f24b5411d5d489c0c17b5912
I20260807 10:56:24.931668 5244 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.73.254" port: 37305 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 10:56:24 dist-test-slave-x5fp krb5kdc[5016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100184, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.253@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:56:24.959501 5247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:24.934657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44640 (local address 127.3.73.254:37305)
0807 10:56:24.935267 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:24.935271 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:24.936407 (+ 1136us) server_negotiation.cc:407] Connection header received
0807 10:56:24.938535 (+ 2128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:24.938541 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:24.938641 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:24.938771 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:24.940897 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:24.942262 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:24.944787 (+ 2525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:24.945581 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:24.945754 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:24.951350 (+ 5596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:24.951356 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:24.951360 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:56:24.951416 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:24.954997 (+ 3581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:24.956381 (+ 1384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:24.956387 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:24.956390 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:24.956552 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:24.957269 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:24.957273 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:24.957275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:24.957606 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:24.957737 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:24.958973 (+ 1236us) server_negotiation.cc:299] Negotiation successful
0807 10:56:24.959184 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":362,"threads_started":1}
I20260807 10:56:24.961440 5249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:24.935359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:37305 (local address 127.0.0.1:44640)
0807 10:56:24.936228 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:24.936269 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 10:56:24.938225 (+ 1956us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:56:24.939038 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:56:24.939047 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:56:24.939626 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:56:24.940599 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:56:24.940627 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:56:24.942496 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:56:24.942502 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:56:24.944523 (+ 2021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:56:24.944542 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:56:24.945910 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:56:24.945917 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:56:24.946333 (+ 416us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:24.947321 (+ 988us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:56:24.947352 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:56:24.950931 (+ 3579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:56:24.955233 (+ 4302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:56:24.955248 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:56:24.955274 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:56:24.956162 (+ 888us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:56:24.956747 (+ 585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:56:24.956756 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:56:24.956763 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:56:24.957109 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:56:24.957765 (+ 656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:56:24.957778 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:56:24.958388 (+ 610us) client_negotiation.cc:769] Sending connection context
0807 10:56:24.958858 (+ 470us) client_negotiation.cc:240] Negotiation successful
0807 10:56:24.959562 (+ 704us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":379,"threads_started":1}
I20260807 10:56:24.963533 5244 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39573 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.3.73.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.73.252@KRBTEST.COM" created.
Aug 07 10:56:24 dist-test-slave-x5fp krb5kdc[5016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100184, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.253@KRBTEST.COM for kudu/127.3.73.253@KRBTEST.COM
I20260807 10:56:24.984202 5251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:24.964741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38566 (local address 127.3.73.253:39573)
0807 10:56:24.965446 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:24.965453 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:24.965480 (+ 27us) server_negotiation.cc:407] Connection header received
0807 10:56:24.965532 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:24.965538 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:24.965736 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:24.965914 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:24.966767 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:24.967989 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:24.969288 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:24.970023 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:24.970187 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:24.977575 (+ 7388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:24.977593 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:24.977612 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:56:24.977657 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:24.980837 (+ 3180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:24.981825 (+ 988us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:24.981834 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:24.981836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:24.981979 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:24.982423 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:24.982426 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:24.982428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:24.982814 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:24.982995 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:24.983499 (+ 504us) server_negotiation.cc:299] Negotiation successful
0807 10:56:24.983852 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":166,"server-negotiator.queue_time_us":540,"thread_start_us":379,"threads_started":1}
I20260807 10:56:24.984236 5249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:24.964282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:39573 (local address 127.0.0.1:38566)
0807 10:56:24.964542 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:24.964562 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 10:56:24.964662 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:56:24.965909 (+ 1247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:56:24.965913 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:56:24.966425 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:56:24.966568 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:56:24.966577 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:56:24.968137 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:56:24.968144 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:56:24.969141 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:56:24.969154 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:56:24.970202 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:56:24.970208 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:56:24.970417 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:24.971271 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:56:24.971290 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:56:24.977234 (+ 5944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:56:24.981025 (+ 3791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:56:24.981034 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:56:24.981041 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:56:24.981661 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:56:24.982120 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:56:24.982127 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:56:24.982133 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:56:24.982309 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:56:24.982958 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:56:24.982965 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:56:24.983168 (+ 203us) client_negotiation.cc:769] Sending connection context
0807 10:56:24.983338 (+ 170us) client_negotiation.cc:240] Negotiation successful
0807 10:56:24.983716 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":6}
I20260807 10:56:24.987313 5244 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42165 } has no permanent_uuid. Determining permanent_uuid...
I20260807 10:56:24.988685 5249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:24.988165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:42165 (local address 127.0.0.1:52508)
0807 10:56:24.988385 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:24.988516 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:42165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260807 10:56:24.989214 5170 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.73.252:42165: connect: Connection refused (error 111)
I20260807 10:56:24.992229 5249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:24.991818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:42165 (local address 127.0.0.1:52520)
0807 10:56:24.992001 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:24.992094 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:42165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 10:56:24.992630 5244 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.252:42165: Network error: Client connection negotiation failed: client connection to 127.3.73.252:42165: connect: Connection refused (error 111)
I20260807 10:56:25.025208 5132 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39573 } attempt: 5
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.73.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.252.keytab.
Entry for principal kudu/127.3.73.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.252.keytab.
W20260807 10:56:25.028692 3367 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.3.73.252: real 0.101s user 0.000s sys 0.040s
I20260807 10:56:25.029960 3367 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskRBZOut/build/asan/bin/kudu
/tmp/dist-test-taskRBZOut/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.252:42165
--webserver_interface=127.3.73.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.252.keytab
--principal=kudu/127.3.73.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.73.212:34325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.73.254:37305,127.3.73.253:39573,127.3.73.252:42165
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 10:56:25 dist-test-slave-x5fp krb5kdc[5016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100184, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for kudu/127.3.73.253@KRBTEST.COM
I20260807 10:56:25.042081 5244 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42165 } attempt: 1
I20260807 10:56:25.043414 5249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:25.042735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:42165 (local address 127.0.0.1:52536)
0807 10:56:25.043080 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:25.043201 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:42165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225}
I20260807 10:56:25.043696 5251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:25.026290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38582 (local address 127.3.73.253:39573)
0807 10:56:25.026490 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:25.026496 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:25.026550 (+ 54us) server_negotiation.cc:407] Connection header received
0807 10:56:25.026799 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:25.026804 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:25.026909 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:25.027019 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:25.028221 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:25.029282 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:25.031208 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:25.031774 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:25.031893 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:25.036838 (+ 4945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:25.036846 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:25.036849 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:56:25.036894 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:25.040122 (+ 3228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:25.041104 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:25.041118 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:25.041121 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:25.041265 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:25.041724 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:25.041730 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:25.041736 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:25.042176 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:25.042363 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:25.043008 (+ 645us) server_negotiation.cc:299] Negotiation successful
0807 10:56:25.043280 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 10:56:25.043797 5135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:25.026206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:39573 (local address 127.0.0.1:38582)
0807 10:56:25.026452 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:25.026477 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 10:56:25.026625 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:56:25.027162 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:56:25.027168 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:56:25.027756 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:56:25.028016 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:56:25.028030 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:56:25.029520 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:56:25.029526 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:56:25.031010 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:56:25.031033 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:56:25.031970 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:56:25.031979 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:56:25.032566 (+ 587us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:25.033424 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:56:25.033457 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:56:25.036472 (+ 3015us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:56:25.040358 (+ 3886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:56:25.040365 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:56:25.040370 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:56:25.040945 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:56:25.041417 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:56:25.041423 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:56:25.041427 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:56:25.041593 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:56:25.042383 (+ 790us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:56:25.042392 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:56:25.042619 (+ 227us) client_negotiation.cc:769] Sending connection context
0807 10:56:25.042826 (+ 207us) client_negotiation.cc:240] Negotiation successful
0807 10:56:25.043263 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94}
I20260807 10:56:25.044983 5249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:25.044300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:42165 (local address 127.0.0.1:52552)
0807 10:56:25.044568 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:25.044689 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:42165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":378,"threads_started":1}
W20260807 10:56:25.045436 5244 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.252:42165: Network error: Client connection negotiation failed: client connection to 127.3.73.252:42165: connect: Connection refused (error 111)
I20260807 10:56:25.045946 5132 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42165 } has no permanent_uuid. Determining permanent_uuid...
I20260807 10:56:25.047127 5135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:25.046665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:42165 (local address 127.0.0.1:52564)
0807 10:56:25.046865 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:25.046966 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:42165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260807 10:56:25.048205 5135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:25.047843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:42165 (local address 127.0.0.1:52566)
0807 10:56:25.048012 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:25.048097 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:42165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 10:56:25.048584 5132 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.252:42165: Network error: Client connection negotiation failed: client connection to 127.3.73.252:42165: connect: Connection refused (error 111)
I20260807 10:56:25.099915 5132 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42165 } attempt: 1
I20260807 10:56:25.101465 5135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:25.100894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:42165 (local address 127.0.0.1:52582)
0807 10:56:25.101157 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:25.101256 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:42165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260807 10:56:25.102686 5135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:25.102290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:42165 (local address 127.0.0.1:52592)
0807 10:56:25.102481 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:25.102560 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:42165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260807 10:56:25.102926 5132 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.252:42165: Network error: Client connection negotiation failed: client connection to 127.3.73.252:42165: connect: Connection refused (error 111)
I20260807 10:56:25.113687 5244 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42165 } attempt: 2
I20260807 10:56:25.114954 5249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:25.114433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:42165 (local address 127.0.0.1:52598)
0807 10:56:25.114650 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:25.114754 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:42165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260807 10:56:25.116104 5249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:25.115694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:42165 (local address 127.0.0.1:52606)
0807 10:56:25.115881 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:25.115962 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:42165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260807 10:56:25.116360 5244 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.252:42165: Network error: Client connection negotiation failed: client connection to 127.3.73.252:42165: connect: Connection refused (error 111)
I20260807 10:56:25.171267 5132 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42165 } attempt: 2
I20260807 10:56:25.172801 5135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:25.172064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:42165 (local address 127.0.0.1:52620)
0807 10:56:25.172435 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:25.172547 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:42165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260807 10:56:25.174050 5135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:25.173631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:42165 (local address 127.0.0.1:52634)
0807 10:56:25.173824 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:25.173916 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:42165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260807 10:56:25.174374 5132 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.252:42165: Network error: Client connection negotiation failed: client connection to 127.3.73.252:42165: connect: Connection refused (error 111)
I20260807 10:56:25.207581 5244 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42165 } attempt: 3
I20260807 10:56:25.209025 5249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:25.208432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:42165 (local address 127.0.0.1:52640)
0807 10:56:25.208666 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:25.208838 (+ 172us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:42165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260807 10:56:25.210275 5249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:25.209761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:42165 (local address 127.0.0.1:52646)
0807 10:56:25.210070 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:25.210149 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:42165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179}
W20260807 10:56:25.210516 5244 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.252:42165: Network error: Client connection negotiation failed: client connection to 127.3.73.252:42165: connect: Connection refused (error 111)
I20260807 10:56:25.280650 5132 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42165 } attempt: 3
I20260807 10:56:25.282084 5135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:25.281460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:42165 (local address 127.0.0.1:52660)
0807 10:56:25.281726 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:25.281848 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:42165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260807 10:56:25.283389 5135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:25.282900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:42165 (local address 127.0.0.1:52674)
0807 10:56:25.283135 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:25.283235 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:42165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
W20260807 10:56:25.283635 5132 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.252:42165: Network error: Client connection negotiation failed: client connection to 127.3.73.252:42165: connect: Connection refused (error 111)
I20260807 10:56:25.353767 5244 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42165 } attempt: 4
I20260807 10:56:25.355052 5249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:25.354523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:42165 (local address 127.0.0.1:52690)
0807 10:56:25.354772 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:25.354878 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:42165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260807 10:56:25.356107 5249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:25.355758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:42165 (local address 127.0.0.1:52700)
0807 10:56:25.355911 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:25.355991 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:42165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 10:56:25.356410 5244 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.252:42165: Network error: Client connection negotiation failed: client connection to 127.3.73.252:42165: connect: Connection refused (error 111)
W20260807 10:56:25.418258 5259 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 10:56:25.418646 5259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 10:56:25.418828 5259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 10:56:25.418959 5259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 10:56:25.426621 5259 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 10:56:25.426738 5259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 10:56:25.426839 5259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 10:56:25.426926 5259 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 10:56:25.427024 5259 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 10:56:25.437975 5259 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.73.212:34325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.73.254:37305,127.3.73.253:39573,127.3.73.252:42165
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/krb5kdc/kudu_127.3.73.252.keytab
--principal=kudu/127.3.73.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.73.252:42165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.73.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 07 Aug 2026 09:12:27 UTC on ccycloud.quasar-rsyjqh.root.comops.site
build id 4572007
ASAN enabled
I20260807 10:56:25.439450 5259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 10:56:25.441023 5259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 10:56:25.454564 5264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:56:25.454507 5265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 10:56:25.455044 5267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 10:56:25.456070 5259 server_base.cc:1034] running on GCE node
I20260807 10:56:25.457141 5259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 10:56:25.458949 5259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 10:56:25.460218 5259 hybrid_clock.cc:648] HybridClock initialized: now 1786100185460140 us; error 30 us; skew 500 ppm
I20260807 10:56:25.460784 5259 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260807 10:56:25.460809 5132 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42165 } attempt: 4
I20260807 10:56:25.462102 5135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:25.461543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:42165 (local address 127.0.0.1:52712)
0807 10:56:25.461793 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:25.461912 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:42165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260807 10:56:25.463399 5135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:25.462920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:42165 (local address 127.0.0.1:52714)
0807 10:56:25.463109 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:25.463224 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.73.252:42165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 10:56:25.463685 5132 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.73.252:42165: Network error: Client connection negotiation failed: client connection to 127.3.73.252:42165: connect: Connection refused (error 111)
Aug 07 10:56:25 dist-test-slave-x5fp krb5kdc[5016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100185, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 10:56:25.465543 5259 init.cc:377] Logged in from keytab as kudu/127.3.73.252@KRBTEST.COM (short username kudu)
I20260807 10:56:25.467422 5259 webserver.cc:460] Webserver started at http://127.3.73.252:34505/ using document root <none> and password file <none>
I20260807 10:56:25.468344 5259 fs_manager.cc:359] Metadata directory not provided
I20260807 10:56:25.468465 5259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 10:56:25.468984 5259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 10:56:25.473681 5259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-2/data/instance:
uuid: "c2352a20953f4d10b9fbb3009d16713b"
format_stamp: "Formatted at 2026-08-07 10:56:25 on dist-test-slave-x5fp"
server_key: "942cfadf125072d667a20a9f1183b4b1"
server_key_iv: "1f14246ad4216d2061f356228fc1dcaa"
server_key_version: "encryptionkey@0"
I20260807 10:56:25.474812 5259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-2/wal/instance:
uuid: "c2352a20953f4d10b9fbb3009d16713b"
format_stamp: "Formatted at 2026-08-07 10:56:25 on dist-test-slave-x5fp"
server_key: "942cfadf125072d667a20a9f1183b4b1"
server_key_iv: "1f14246ad4216d2061f356228fc1dcaa"
server_key_version: "encryptionkey@0"
I20260807 10:56:25.481536 5259 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260807 10:56:25.485811 5276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 10:56:25.487648 5259 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 10:56:25.487895 5259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-2/data,/tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-2/wal
uuid: "c2352a20953f4d10b9fbb3009d16713b"
format_stamp: "Formatted at 2026-08-07 10:56:25 on dist-test-slave-x5fp"
server_key: "942cfadf125072d667a20a9f1183b4b1"
server_key_iv: "1f14246ad4216d2061f356228fc1dcaa"
server_key_version: "encryptionkey@0"
I20260807 10:56:25.488132 5259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 10:56:25.504580 5259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 10:56:25.509758 5259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 10:56:25.510124 5259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 10:56:25.531662 5259 rpc_server.cc:307] RPC server started. Bound to: 127.3.73.252:42165
I20260807 10:56:25.531694 5361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.73.252:42165 every 8 connection(s)
I20260807 10:56:25.533252 5259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-2/data/info.pb
I20260807 10:56:25.534601 3367 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskRBZOut/build/asan/bin/kudu as pid 5259
I20260807 10:56:25.534868 3367 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskRBZOut/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786099767749326-3367-0/minicluster-data/master-2/wal/instance
I20260807 10:56:25.535442 3367 external_mini_cluster.cc:1428] Setting key be06d0f5387a58fc4d8820b53ba99e9b
I20260807 10:56:25.538800 5362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 10:56:25.549918 5362 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.73.254" port: 37305 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 10:56:25 dist-test-slave-x5fp krb5kdc[5016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.252@KRBTEST.COM
Aug 07 10:56:25 dist-test-slave-x5fp krb5kdc[5016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100185, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.252@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
I20260807 10:56:25.566622 5367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:25.537621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52716 (local address 127.3.73.252:42165)
0807 10:56:25.538484 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:25.538496 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:25.538541 (+ 45us) server_negotiation.cc:407] Connection header received
0807 10:56:25.540399 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:25.540424 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:25.540949 (+ 525us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:25.541331 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:25.544456 (+ 3125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:25.546273 (+ 1817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:25.548288 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:25.549182 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:25.549340 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:25.555480 (+ 6140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:25.555497 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:25.555519 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:56:25.555586 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:25.560342 (+ 4756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:25.561543 (+ 1201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:25.561552 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:25.561560 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:25.561908 (+ 348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:25.562500 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:25.562509 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:25.562514 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:25.563263 (+ 749us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:25.563674 (+ 411us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:25.564313 (+ 639us) server_negotiation.cc:299] Negotiation successful
0807 10:56:25.565126 (+ 813us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":419,"threads_started":1}
I20260807 10:56:25.573508 5369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:25.554026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44652 (local address 127.3.73.254:37305)
0807 10:56:25.555926 (+ 1900us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:25.555933 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:25.556079 (+ 146us) server_negotiation.cc:407] Connection header received
0807 10:56:25.556170 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:25.556174 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:25.556275 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:25.556547 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:25.558411 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:25.559594 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:25.561703 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:25.562382 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:25.562569 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:25.566229 (+ 3660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:25.566239 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:25.566244 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:56:25.566297 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:25.569748 (+ 3451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:25.570911 (+ 1163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:25.570920 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:25.570927 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:25.571090 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:25.571549 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:25.571554 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:25.571556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:25.571856 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:25.572029 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:25.572873 (+ 844us) server_negotiation.cc:299] Negotiation successful
0807 10:56:25.573145 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1744,"thread_start_us":1531,"threads_started":1}
I20260807 10:56:25.573534 5368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:25.553890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.254:37305 (local address 127.0.0.1:44652)
0807 10:56:25.555434 (+ 1544us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:25.555519 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0807 10:56:25.555764 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:56:25.556753 (+ 989us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:56:25.556769 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:56:25.557715 (+ 946us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:56:25.558126 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:56:25.558144 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:56:25.559816 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:56:25.559823 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:56:25.561503 (+ 1680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:56:25.561531 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:56:25.562642 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:56:25.562648 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:56:25.562842 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:25.563591 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:56:25.563616 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:56:25.565949 (+ 2333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:56:25.569949 (+ 4000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:56:25.569960 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:56:25.569966 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:56:25.570755 (+ 789us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:56:25.571229 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:56:25.571236 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:56:25.571239 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:56:25.571406 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:56:25.572012 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:56:25.572021 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:56:25.572281 (+ 260us) client_negotiation.cc:769] Sending connection context
0807 10:56:25.572555 (+ 274us) client_negotiation.cc:240] Negotiation successful
0807 10:56:25.573014 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1068,"mutex_wait_us":3,"thread_start_us":591,"threads_started":1}
I20260807 10:56:25.575924 5362 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39573 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 10:56:25 dist-test-slave-x5fp krb5kdc[5016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.254@KRBTEST.COM
Aug 07 10:56:25 dist-test-slave-x5fp krb5kdc[5016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100185, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.252@KRBTEST.COM for kudu/127.3.73.253@KRBTEST.COM
I20260807 10:56:25.590046 5370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:25.570636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44658 (local address 127.3.73.254:37305)
0807 10:56:25.571736 (+ 1100us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:25.571741 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:25.571765 (+ 24us) server_negotiation.cc:407] Connection header received
0807 10:56:25.571877 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:25.571881 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:25.572000 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:25.572099 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:25.574026 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:25.575303 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:25.577390 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:25.578016 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:25.578182 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:25.582889 (+ 4707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:25.582895 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:25.582899 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:56:25.582939 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:25.586336 (+ 3397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:25.587402 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:25.587407 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:25.587409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:25.587622 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:25.588148 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:25.588152 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:25.588154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:25.588459 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:25.588701 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:25.589257 (+ 556us) server_negotiation.cc:299] Negotiation successful
0807 10:56:25.589523 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":945,"thread_start_us":640,"threads_started":1}
I20260807 10:56:25.592667 5368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:25.576899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.253:39573 (local address 127.0.0.1:38586)
0807 10:56:25.577277 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:25.577300 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 10:56:25.577424 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:56:25.578013 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:56:25.578018 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:56:25.578558 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:56:25.578751 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:56:25.578762 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:56:25.579958 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:56:25.579965 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:56:25.581046 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:56:25.581062 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:56:25.581888 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:56:25.581895 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:56:25.582137 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:25.582837 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:56:25.582859 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:56:25.585440 (+ 2581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:56:25.589555 (+ 4115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:56:25.589561 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:56:25.589565 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:56:25.590189 (+ 624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:56:25.590666 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:56:25.590670 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:56:25.590672 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:56:25.590833 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:56:25.591550 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:56:25.591554 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:56:25.591740 (+ 186us) client_negotiation.cc:769] Sending connection context
0807 10:56:25.591881 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 10:56:25.592268 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221}
I20260807 10:56:25.593148 5371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:25.577088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38586 (local address 127.3.73.253:39573)
0807 10:56:25.577670 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:25.577675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:25.577697 (+ 22us) server_negotiation.cc:407] Connection header received
0807 10:56:25.577752 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:25.577756 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:25.577821 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:25.577926 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:25.578998 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:25.579778 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:25.581234 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:25.581691 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:25.581863 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:25.585714 (+ 3851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:25.585720 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:25.585724 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:56:25.585772 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:25.589361 (+ 3589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:25.590382 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:25.590388 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:25.590390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:25.590536 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:25.590974 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:25.590982 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:25.590986 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:25.591413 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:25.591539 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:25.592073 (+ 534us) server_negotiation.cc:299] Negotiation successful
0807 10:56:25.592449 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":340,"threads_started":1}
I20260807 10:56:25.594381 5362 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42165 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 10:56:25 dist-test-slave-x5fp krb5kdc[5016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100185, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.252@KRBTEST.COM for kudu/127.3.73.252@KRBTEST.COM
Aug 07 10:56:25 dist-test-slave-x5fp krb5kdc[5016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.73.253@KRBTEST.COM
I20260807 10:56:25.611094 5368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:25.595206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:42165 (local address 127.0.0.1:52732)
0807 10:56:25.595383 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:25.595404 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 10:56:25.595516 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:56:25.595993 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:56:25.596003 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:56:25.596532 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:56:25.596742 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:56:25.596758 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:56:25.598399 (+ 1641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:56:25.598403 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:56:25.599770 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:56:25.599783 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:56:25.600846 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:56:25.600853 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:56:25.601006 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:25.601660 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:56:25.601680 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:56:25.604213 (+ 2533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:56:25.607733 (+ 3520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:56:25.607741 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:56:25.607748 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:56:25.608366 (+ 618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:56:25.608914 (+ 548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:56:25.608920 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:56:25.608923 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:56:25.609119 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:56:25.609855 (+ 736us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:56:25.609859 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:56:25.610087 (+ 228us) client_negotiation.cc:769] Sending connection context
0807 10:56:25.610277 (+ 190us) client_negotiation.cc:240] Negotiation successful
0807 10:56:25.610603 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260807 10:56:25.611337 5367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:25.595495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52732 (local address 127.3.73.252:42165)
0807 10:56:25.595700 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:25.595706 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:25.595731 (+ 25us) server_negotiation.cc:407] Connection header received
0807 10:56:25.595772 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:25.595775 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:25.595840 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:25.595918 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:25.597084 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:25.598235 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:25.599944 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:25.600554 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:25.600697 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:25.604425 (+ 3728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:25.604430 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:25.604434 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:56:25.604465 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:25.607567 (+ 3102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:25.608525 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:25.608530 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:25.608532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:25.608674 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:25.609258 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:25.609263 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:25.609268 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:25.609687 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:25.609867 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:25.610439 (+ 572us) server_negotiation.cc:299] Negotiation successful
0807 10:56:25.610866 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":80,"server-negotiator.queue_time_us":75}
I20260807 10:56:25.612780 5371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:25.592356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38592 (local address 127.3.73.253:39573)
0807 10:56:25.593688 (+ 1332us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:25.593693 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:25.593706 (+ 13us) server_negotiation.cc:407] Connection header received
0807 10:56:25.593747 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:25.593751 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:25.593806 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:25.593933 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:25.595702 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:25.596784 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:25.598748 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:25.599418 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:25.599569 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:25.605379 (+ 5810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:25.605385 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:25.605388 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:56:25.605429 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:25.608953 (+ 3524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:25.609974 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:25.609981 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:25.609985 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:25.610178 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:25.610741 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:25.610748 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:25.610753 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:25.611145 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:25.611428 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:25.612079 (+ 651us) server_negotiation.cc:299] Negotiation successful
0807 10:56:25.612333 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1253,"thread_start_us":1908,"threads_started":1}
I20260807 10:56:25.617631 5362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2352a20953f4d10b9fbb3009d16713b: Bootstrap starting.
I20260807 10:56:25.618145 3367 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 10:56:25.621277 5362 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2352a20953f4d10b9fbb3009d16713b: Neither blocks nor log segments found. Creating new log.
I20260807 10:56:25.622419 5362 log.cc:824] T 00000000000000000000000000000000 P c2352a20953f4d10b9fbb3009d16713b: Log is configured to *not* fsync() on all Append() calls
I20260807 10:56:25.620635 5098 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "a048b4545549414a9f6b3757cd1416c9"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44658
I20260807 10:56:25.625623 5362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2352a20953f4d10b9fbb3009d16713b: No bootstrap required, opened a new log
I20260807 10:56:25.633656 5362 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2352a20953f4d10b9fbb3009d16713b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a048b4545549414a9f6b3757cd1416c9" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 37305 } } peers { permanent_uuid: "6f22e95eb2cd421caa9325952c2edda2" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39573 } } peers { permanent_uuid: "c2352a20953f4d10b9fbb3009d16713b" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42165 } }
I20260807 10:56:25.634243 5362 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2352a20953f4d10b9fbb3009d16713b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2352a20953f4d10b9fbb3009d16713b, State: Initialized, Role: FOLLOWER
I20260807 10:56:25.634909 5362 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2352a20953f4d10b9fbb3009d16713b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a048b4545549414a9f6b3757cd1416c9" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 37305 } } peers { permanent_uuid: "6f22e95eb2cd421caa9325952c2edda2" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39573 } } peers { permanent_uuid: "c2352a20953f4d10b9fbb3009d16713b" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42165 } }
I20260807 10:56:25.635993 5375 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2352a20953f4d10b9fbb3009d16713b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a048b4545549414a9f6b3757cd1416c9" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 37305 } } peers { permanent_uuid: "6f22e95eb2cd421caa9325952c2edda2" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39573 } } peers { permanent_uuid: "c2352a20953f4d10b9fbb3009d16713b" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42165 } } }
I20260807 10:56:25.636389 5375 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2352a20953f4d10b9fbb3009d16713b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 10:56:25.637313 5362 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2352a20953f4d10b9fbb3009d16713b [sys.catalog]: configured and running, proceeding with master startup.
W20260807 10:56:25.648123 5391 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c2352a20953f4d10b9fbb3009d16713b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 10:56:25.648320 5391 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 10:56:25.655679 5244 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42165 } attempt: 5
I20260807 10:56:25.645979 5259 master_runner.cc:186] Error getting master registration for 127.3.73.254:37305: OK, instance_id { permanent_uuid: "a048b4545549414a9f6b3757cd1416c9" instance_seqno: 1786100184310805 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 10:56:25.657936 5259 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 07 10:56:25 dist-test-slave-x5fp krb5kdc[5016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100184, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.253@KRBTEST.COM for kudu/127.3.73.252@KRBTEST.COM
I20260807 10:56:25.675524 5249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:25.656521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:42165 (local address 127.0.0.1:52734)
0807 10:56:25.656973 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:25.657002 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0807 10:56:25.657153 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:56:25.658264 (+ 1111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:56:25.658268 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:56:25.658677 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:56:25.658859 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:56:25.658868 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:56:25.660402 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:56:25.660408 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:56:25.661910 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:56:25.661927 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:56:25.663019 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:56:25.663028 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:56:25.663290 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:25.664069 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:56:25.664094 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:56:25.667263 (+ 3169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:56:25.672019 (+ 4756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:56:25.672026 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:56:25.672030 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:56:25.672774 (+ 744us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:56:25.673323 (+ 549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:56:25.673330 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:56:25.673334 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:56:25.673549 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:56:25.674370 (+ 821us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:56:25.674376 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:56:25.674559 (+ 183us) client_negotiation.cc:769] Sending connection context
0807 10:56:25.674741 (+ 182us) client_negotiation.cc:240] Negotiation successful
0807 10:56:25.675099 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283}
I20260807 10:56:25.675544 5367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:25.657614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52734 (local address 127.3.73.252:42165)
0807 10:56:25.657866 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:25.657873 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:25.657898 (+ 25us) server_negotiation.cc:407] Connection header received
0807 10:56:25.657970 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:25.657977 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:25.658061 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:25.658204 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:25.659026 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:25.660211 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:25.662087 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:25.662837 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:25.662988 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:25.667650 (+ 4662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:25.667659 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:25.667665 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:56:25.667729 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:25.671790 (+ 4061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:25.672957 (+ 1167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:25.672963 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:25.672967 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:25.673145 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:25.673688 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:25.673699 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:25.673702 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:25.674219 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:25.674372 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:25.674893 (+ 521us) server_negotiation.cc:299] Negotiation successful
0807 10:56:25.675163 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260807 10:56:25.682448 5244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f22e95eb2cd421caa9325952c2edda2: Bootstrap starting.
I20260807 10:56:25.687224 5244 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f22e95eb2cd421caa9325952c2edda2: Neither blocks nor log segments found. Creating new log.
I20260807 10:56:25.688838 5244 log.cc:824] T 00000000000000000000000000000000 P 6f22e95eb2cd421caa9325952c2edda2: Log is configured to *not* fsync() on all Append() calls
I20260807 10:56:25.693185 5244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f22e95eb2cd421caa9325952c2edda2: No bootstrap required, opened a new log
I20260807 10:56:25.705528 5244 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f22e95eb2cd421caa9325952c2edda2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a048b4545549414a9f6b3757cd1416c9" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 37305 } } peers { permanent_uuid: "6f22e95eb2cd421caa9325952c2edda2" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39573 } } peers { permanent_uuid: "c2352a20953f4d10b9fbb3009d16713b" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42165 } }
I20260807 10:56:25.706060 5244 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f22e95eb2cd421caa9325952c2edda2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f22e95eb2cd421caa9325952c2edda2, State: Initialized, Role: FOLLOWER
I20260807 10:56:25.706676 5244 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f22e95eb2cd421caa9325952c2edda2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a048b4545549414a9f6b3757cd1416c9" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 37305 } } peers { permanent_uuid: "6f22e95eb2cd421caa9325952c2edda2" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39573 } } peers { permanent_uuid: "c2352a20953f4d10b9fbb3009d16713b" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42165 } }
I20260807 10:56:25.707885 5396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f22e95eb2cd421caa9325952c2edda2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a048b4545549414a9f6b3757cd1416c9" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 37305 } } peers { permanent_uuid: "6f22e95eb2cd421caa9325952c2edda2" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39573 } } peers { permanent_uuid: "c2352a20953f4d10b9fbb3009d16713b" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42165 } } }
I20260807 10:56:25.708360 5396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f22e95eb2cd421caa9325952c2edda2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 10:56:25.709473 5244 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f22e95eb2cd421caa9325952c2edda2 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 10:56:25.724107 5412 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6f22e95eb2cd421caa9325952c2edda2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 10:56:25.724339 5412 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 10:56:25.721145 5145 master_runner.cc:186] Error getting master registration for 127.3.73.254:37305: OK, instance_id { permanent_uuid: "a048b4545549414a9f6b3757cd1416c9" instance_seqno: 1786100184310805 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 10:56:25.733559 5145 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 10:56:25.740942 5132 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42165 } attempt: 5
Aug 07 10:56:25 dist-test-slave-x5fp krb5kdc[5016](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786100184, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.73.254@KRBTEST.COM for kudu/127.3.73.252@KRBTEST.COM
I20260807 10:56:25.759011 5135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:25.741790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.73.252:42165 (local address 127.0.0.1:52738)
0807 10:56:25.742116 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0807 10:56:25.742137 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 10:56:25.742273 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 10:56:25.742915 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 10:56:25.742921 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 10:56:25.743390 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0807 10:56:25.743568 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:56:25.743579 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:56:25.745265 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:56:25.745275 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:56:25.746871 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 10:56:25.746892 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 10:56:25.747779 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 10:56:25.747786 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 10:56:25.748006 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:25.748833 (+ 827us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 10:56:25.748858 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 10:56:25.752027 (+ 3169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 10:56:25.755768 (+ 3741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:56:25.755776 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:56:25.755780 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:56:25.756369 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:56:25.756900 (+ 531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 10:56:25.756918 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 10:56:25.756921 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 10:56:25.757087 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 10:56:25.757868 (+ 781us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 10:56:25.757873 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 10:56:25.758050 (+ 177us) client_negotiation.cc:769] Sending connection context
0807 10:56:25.758212 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 10:56:25.758561 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186}
I20260807 10:56:25.759018 5367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:25.741874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52738 (local address 127.3.73.252:42165)
0807 10:56:25.742532 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:25.742539 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:25.742563 (+ 24us) server_negotiation.cc:407] Connection header received
0807 10:56:25.742635 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:25.742641 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:25.742753 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:25.742857 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:25.743770 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:25.745025 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:25.747045 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:25.747598 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:25.747759 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:25.752291 (+ 4532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:25.752296 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:25.752299 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:56:25.752340 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:25.755558 (+ 3218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:25.756545 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:25.756553 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:25.756559 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:25.756776 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:25.757226 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:25.757234 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:25.757237 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:25.757637 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:25.757787 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:25.758348 (+ 561us) server_negotiation.cc:299] Negotiation successful
0807 10:56:25.758616 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501}
W20260807 10:56:25.762557 5288 tablet.cc:2369] T 00000000000000000000000000000000 P c2352a20953f4d10b9fbb3009d16713b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:56:25.766119 5132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a048b4545549414a9f6b3757cd1416c9: Bootstrap starting.
I20260807 10:56:25.770954 5132 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a048b4545549414a9f6b3757cd1416c9: Neither blocks nor log segments found. Creating new log.
I20260807 10:56:25.772439 5132 log.cc:824] T 00000000000000000000000000000000 P a048b4545549414a9f6b3757cd1416c9: Log is configured to *not* fsync() on all Append() calls
I20260807 10:56:25.776489 5132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a048b4545549414a9f6b3757cd1416c9: No bootstrap required, opened a new log
I20260807 10:56:25.784864 5132 raft_consensus.cc:348] T 00000000000000000000000000000000 P a048b4545549414a9f6b3757cd1416c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a048b4545549414a9f6b3757cd1416c9" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 37305 } } peers { permanent_uuid: "6f22e95eb2cd421caa9325952c2edda2" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39573 } } peers { permanent_uuid: "c2352a20953f4d10b9fbb3009d16713b" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42165 } }
I20260807 10:56:25.785403 5132 raft_consensus.cc:729] T 00000000000000000000000000000000 P a048b4545549414a9f6b3757cd1416c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a048b4545549414a9f6b3757cd1416c9, State: Initialized, Role: FOLLOWER
I20260807 10:56:25.785991 5132 consensus_queue.cc:260] T 00000000000000000000000000000000 P a048b4545549414a9f6b3757cd1416c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a048b4545549414a9f6b3757cd1416c9" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 37305 } } peers { permanent_uuid: "6f22e95eb2cd421caa9325952c2edda2" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39573 } } peers { permanent_uuid: "c2352a20953f4d10b9fbb3009d16713b" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42165 } }
I20260807 10:56:25.787433 5415 sys_catalog.cc:455] T 00000000000000000000000000000000 P a048b4545549414a9f6b3757cd1416c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a048b4545549414a9f6b3757cd1416c9" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 37305 } } peers { permanent_uuid: "6f22e95eb2cd421caa9325952c2edda2" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39573 } } peers { permanent_uuid: "c2352a20953f4d10b9fbb3009d16713b" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42165 } } }
I20260807 10:56:25.788196 5415 sys_catalog.cc:458] T 00000000000000000000000000000000 P a048b4545549414a9f6b3757cd1416c9 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 10:56:25.788887 5132 sys_catalog.cc:564] T 00000000000000000000000000000000 P a048b4545549414a9f6b3757cd1416c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 10:56:25.788985 5098 raft_consensus.cc:482] T 00000000000000000000000000000000 P a048b4545549414a9f6b3757cd1416c9 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 10:56:25.789201 5098 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a048b4545549414a9f6b3757cd1416c9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:56:25.793614 5098 raft_consensus.cc:504] T 00000000000000000000000000000000 P a048b4545549414a9f6b3757cd1416c9 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a048b4545549414a9f6b3757cd1416c9" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 37305 } } peers { permanent_uuid: "6f22e95eb2cd421caa9325952c2edda2" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39573 } } peers { permanent_uuid: "c2352a20953f4d10b9fbb3009d16713b" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42165 } }
I20260807 10:56:25.797217 5200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a048b4545549414a9f6b3757cd1416c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6f22e95eb2cd421caa9325952c2edda2"
I20260807 10:56:25.797426 5325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a048b4545549414a9f6b3757cd1416c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c2352a20953f4d10b9fbb3009d16713b"
I20260807 10:56:25.797717 5200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f22e95eb2cd421caa9325952c2edda2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 10:56:25.798043 5325 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2352a20953f4d10b9fbb3009d16713b [term 0 FOLLOWER]: Advancing to term 1
W20260807 10:56:25.799223 5071 tablet.cc:2369] T 00000000000000000000000000000000 P a048b4545549414a9f6b3757cd1416c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:56:25.801836 5200 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6f22e95eb2cd421caa9325952c2edda2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a048b4545549414a9f6b3757cd1416c9 in term 1.
I20260807 10:56:25.802736 5325 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c2352a20953f4d10b9fbb3009d16713b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a048b4545549414a9f6b3757cd1416c9 in term 1.
I20260807 10:56:25.802645 5064 leader_election.cc:302] T 00000000000000000000000000000000 P a048b4545549414a9f6b3757cd1416c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f22e95eb2cd421caa9325952c2edda2, a048b4545549414a9f6b3757cd1416c9; no voters:
I20260807 10:56:25.803795 5098 leader_election.cc:288] T 00000000000000000000000000000000 P a048b4545549414a9f6b3757cd1416c9 [CANDIDATE]: Term 1 election: Requested vote from peers 6f22e95eb2cd421caa9325952c2edda2 (127.3.73.253:39573), c2352a20953f4d10b9fbb3009d16713b (127.3.73.252:42165)
I20260807 10:56:25.804872 5415 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a048b4545549414a9f6b3757cd1416c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 10:56:25.825030 5415 raft_consensus.cc:686] T 00000000000000000000000000000000 P a048b4545549414a9f6b3757cd1416c9 [term 1 LEADER]: Becoming Leader. State: Replica: a048b4545549414a9f6b3757cd1416c9, State: Running, Role: LEADER
I20260807 10:56:25.826097 5415 consensus_queue.cc:237] T 00000000000000000000000000000000 P a048b4545549414a9f6b3757cd1416c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a048b4545549414a9f6b3757cd1416c9" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 37305 } } peers { permanent_uuid: "6f22e95eb2cd421caa9325952c2edda2" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39573 } } peers { permanent_uuid: "c2352a20953f4d10b9fbb3009d16713b" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42165 } }
I20260807 10:56:25.838517 5419 sys_catalog.cc:455] T 00000000000000000000000000000000 P a048b4545549414a9f6b3757cd1416c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a048b4545549414a9f6b3757cd1416c9. Latest consensus state: current_term: 1 leader_uuid: "a048b4545549414a9f6b3757cd1416c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a048b4545549414a9f6b3757cd1416c9" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 37305 } } peers { permanent_uuid: "6f22e95eb2cd421caa9325952c2edda2" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39573 } } peers { permanent_uuid: "c2352a20953f4d10b9fbb3009d16713b" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42165 } } }
I20260807 10:56:25.839215 5419 sys_catalog.cc:458] T 00000000000000000000000000000000 P a048b4545549414a9f6b3757cd1416c9 [sys.catalog]: This master's current role is: LEADER
I20260807 10:56:25.862424 5447 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 10:56:25.880568 5447 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 10:56:25.885779 5370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:25.834619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44662 (local address 127.3.73.254:37305)
0807 10:56:25.834881 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:25.834890 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:25.837965 (+ 3075us) server_negotiation.cc:407] Connection header received
0807 10:56:25.839983 (+ 2018us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:25.839993 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:25.845173 (+ 5180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:25.845325 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:25.857871 (+ 12546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:25.859370 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:25.867047 (+ 7677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:25.867982 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:25.868124 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:25.873929 (+ 5805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:25.873938 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:25.873942 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:56:25.873990 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:25.878218 (+ 4228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:25.881996 (+ 3778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:25.882008 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:25.882014 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:25.882279 (+ 265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:25.882945 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:25.882953 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:25.882959 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:25.883500 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:25.883851 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:25.884766 (+ 915us) server_negotiation.cc:299] Negotiation successful
0807 10:56:25.885157 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260807 10:56:25.900679 5174 tablet.cc:2369] T 00000000000000000000000000000000 P 6f22e95eb2cd421caa9325952c2edda2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 10:56:25.903409 5367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:25.834409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52742 (local address 127.3.73.252:42165)
0807 10:56:25.834659 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:25.834669 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:25.850446 (+ 15777us) server_negotiation.cc:407] Connection header received
0807 10:56:25.857249 (+ 6803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:25.857257 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:25.857358 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:25.857450 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:25.861749 (+ 4299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:25.862818 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:25.865050 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:25.865733 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:25.865838 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:25.885758 (+ 19920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:25.885767 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:25.885772 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:56:25.885833 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:25.890085 (+ 4252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:25.896419 (+ 6334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:25.896433 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:25.896439 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:25.896631 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:25.897209 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:25.897216 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:25.897220 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:25.897692 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:25.897987 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:25.902487 (+ 4500us) server_negotiation.cc:299] Negotiation successful
0807 10:56:25.902906 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 10:56:25.905741 5371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:25.851153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38596 (local address 127.3.73.253:39573)
0807 10:56:25.851344 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:25.851350 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:25.861831 (+ 10481us) server_negotiation.cc:407] Connection header received
0807 10:56:25.862114 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:25.862120 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:25.862221 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:25.862294 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:25.864775 (+ 2481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:25.865853 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:25.869464 (+ 3611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:25.870180 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:25.870300 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:25.891346 (+ 21046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 10:56:25.891355 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 10:56:25.891360 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 10:56:25.891417 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0807 10:56:25.895287 (+ 3870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:25.898838 (+ 3551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:25.898846 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:25.898850 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:25.899062 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 10:56:25.899625 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 10:56:25.899632 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 10:56:25.899636 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 10:56:25.900098 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 10:56:25.901720 (+ 1622us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:25.905033 (+ 3313us) server_negotiation.cc:299] Negotiation successful
0807 10:56:25.905329 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 10:56:25.916481 5325 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c2352a20953f4d10b9fbb3009d16713b [term 1 FOLLOWER]: Refusing update from remote peer a048b4545549414a9f6b3757cd1416c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:56:25.918087 5419 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a048b4545549414a9f6b3757cd1416c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2352a20953f4d10b9fbb3009d16713b" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:56:25.920599 5200 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6f22e95eb2cd421caa9325952c2edda2 [term 1 FOLLOWER]: Refusing update from remote peer a048b4545549414a9f6b3757cd1416c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 10:56:25.924945 5419 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a048b4545549414a9f6b3757cd1416c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f22e95eb2cd421caa9325952c2edda2" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 10:56:25.946190 5396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f22e95eb2cd421caa9325952c2edda2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a048b4545549414a9f6b3757cd1416c9. Latest consensus state: current_term: 1 leader_uuid: "a048b4545549414a9f6b3757cd1416c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a048b4545549414a9f6b3757cd1416c9" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 37305 } } peers { permanent_uuid: "6f22e95eb2cd421caa9325952c2edda2" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39573 } } peers { permanent_uuid: "c2352a20953f4d10b9fbb3009d16713b" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42165 } } }
I20260807 10:56:25.946619 5396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f22e95eb2cd421caa9325952c2edda2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 10:56:25.951254 5455 mvcc.cc:205] Tried to move back new op lower bound from 7315866361481773056 to 7315866361152229376. Current Snapshot: MvccSnapshot[applied={T|T < 7315866361481773056}]
I20260807 10:56:25.961158 5396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f22e95eb2cd421caa9325952c2edda2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a048b4545549414a9f6b3757cd1416c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a048b4545549414a9f6b3757cd1416c9" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 37305 } } peers { permanent_uuid: "6f22e95eb2cd421caa9325952c2edda2" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39573 } } peers { permanent_uuid: "c2352a20953f4d10b9fbb3009d16713b" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42165 } } }
I20260807 10:56:25.961485 5396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f22e95eb2cd421caa9325952c2edda2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 10:56:25.964659 5375 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2352a20953f4d10b9fbb3009d16713b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a048b4545549414a9f6b3757cd1416c9. Latest consensus state: current_term: 1 leader_uuid: "a048b4545549414a9f6b3757cd1416c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a048b4545549414a9f6b3757cd1416c9" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 37305 } } peers { permanent_uuid: "6f22e95eb2cd421caa9325952c2edda2" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39573 } } peers { permanent_uuid: "c2352a20953f4d10b9fbb3009d16713b" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42165 } } }
I20260807 10:56:25.965080 5375 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2352a20953f4d10b9fbb3009d16713b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 10:56:25.976154 5457 mvcc.cc:205] Tried to move back new op lower bound from 7315866361481773056 to 7315866361152229376. Current Snapshot: MvccSnapshot[applied={T|T < 7315866361481773056}]
I20260807 10:56:25.978111 5375 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2352a20953f4d10b9fbb3009d16713b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a048b4545549414a9f6b3757cd1416c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a048b4545549414a9f6b3757cd1416c9" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 37305 } } peers { permanent_uuid: "6f22e95eb2cd421caa9325952c2edda2" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39573 } } peers { permanent_uuid: "c2352a20953f4d10b9fbb3009d16713b" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42165 } } }
I20260807 10:56:25.978401 5375 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2352a20953f4d10b9fbb3009d16713b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 10:56:25.982445 5447 catalog_manager.cc:1347] Generated new cluster ID: 05f1d971fdf540be98c78d55ed3028c5
I20260807 10:56:25.982595 5447 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 10:56:25.984488 5419 sys_catalog.cc:455] T 00000000000000000000000000000000 P a048b4545549414a9f6b3757cd1416c9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a048b4545549414a9f6b3757cd1416c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a048b4545549414a9f6b3757cd1416c9" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 37305 } } peers { permanent_uuid: "6f22e95eb2cd421caa9325952c2edda2" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39573 } } peers { permanent_uuid: "c2352a20953f4d10b9fbb3009d16713b" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42165 } } }
I20260807 10:56:25.984778 5419 sys_catalog.cc:458] T 00000000000000000000000000000000 P a048b4545549414a9f6b3757cd1416c9 [sys.catalog]: This master's current role is: LEADER
I20260807 10:56:25.985044 5419 sys_catalog.cc:455] T 00000000000000000000000000000000 P a048b4545549414a9f6b3757cd1416c9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a048b4545549414a9f6b3757cd1416c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a048b4545549414a9f6b3757cd1416c9" member_type: VOTER last_known_addr { host: "127.3.73.254" port: 37305 } } peers { permanent_uuid: "6f22e95eb2cd421caa9325952c2edda2" member_type: VOTER last_known_addr { host: "127.3.73.253" port: 39573 } } peers { permanent_uuid: "c2352a20953f4d10b9fbb3009d16713b" member_type: VOTER last_known_addr { host: "127.3.73.252" port: 42165 } } }
I20260807 10:56:25.985220 5419 sys_catalog.cc:458] T 00000000000000000000000000000000 P a048b4545549414a9f6b3757cd1416c9 [sys.catalog]: This master's current role is: LEADER
I20260807 10:56:26.004010 5447 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 10:56:26.006848 5447 catalog_manager.cc:1504] Loading token signing keys...
I20260807 10:56:26.022156 5447 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a048b4545549414a9f6b3757cd1416c9: Generated new TSK 0
I20260807 10:56:26.023521 5447 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 10:56:26.192559 5370 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:56:26.168853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44664 (local address 127.3.73.254:37305)
0807 10:56:26.169052 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:26.169062 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:26.190081 (+ 21019us) server_negotiation.cc:407] Connection header received
0807 10:56:26.190424 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:26.190431 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:26.190550 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:26.190631 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:26.192184 (+ 1553us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44664: BlockingRecv error: recv got EOF from 127.0.0.1:44664 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260807 10:56:26.195528 5367 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:56:26.175323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52748 (local address 127.3.73.252:42165)
0807 10:56:26.175572 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:26.175580 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:26.193573 (+ 17993us) server_negotiation.cc:407] Connection header received
0807 10:56:26.193851 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:26.193858 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:26.193952 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:26.194030 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:26.195215 (+ 1185us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52748: BlockingRecv error: recv got EOF from 127.0.0.1:52748 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260807 10:56:26.199352 5371 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:56:26.196970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38612 (local address 127.3.73.253:39573)
0807 10:56:26.197187 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:26.197195 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:26.197628 (+ 433us) server_negotiation.cc:407] Connection header received
0807 10:56:26.197880 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:26.197886 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:26.197981 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:26.198061 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:26.199066 (+ 1005us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38612: BlockingRecv error: recv got EOF from 127.0.0.1:38612 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260807 10:56:26.247092 5371 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:56:26.233555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38624 (local address 127.3.73.253:39573)
0807 10:56:26.233754 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:26.233761 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:26.244622 (+ 10861us) server_negotiation.cc:407] Connection header received
0807 10:56:26.245271 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:26.245278 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:26.245432 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:26.245519 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:26.246776 (+ 1257us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38624: BlockingRecv error: recv got EOF from 127.0.0.1:38624 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260807 10:56:26.298128 5371 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:56:26.281116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38628 (local address 127.3.73.253:39573)
0807 10:56:26.281326 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:26.281335 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:26.296170 (+ 14835us) server_negotiation.cc:407] Connection header received
0807 10:56:26.296426 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:26.296432 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:26.296560 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:26.296666 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:26.297862 (+ 1196us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38628: BlockingRecv error: recv got EOF from 127.0.0.1:38628 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260807 10:56:26.335304 5371 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:56:26.322948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38642 (local address 127.3.73.253:39573)
0807 10:56:26.323156 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:26.323162 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:26.333462 (+ 10300us) server_negotiation.cc:407] Connection header received
0807 10:56:26.333719 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:26.333725 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:26.333825 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:26.333925 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:26.335010 (+ 1085us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38642: BlockingRecv error: recv got EOF from 127.0.0.1:38642 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260807 10:56:26.355517 5371 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:56:26.353013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38658 (local address 127.3.73.253:39573)
0807 10:56:26.353365 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:26.353376 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:26.354001 (+ 625us) server_negotiation.cc:407] Connection header received
0807 10:56:26.354242 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:26.354254 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:26.354337 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:26.354454 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:26.355262 (+ 808us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38658: BlockingRecv error: recv got EOF from 127.0.0.1:38658 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260807 10:56:26.370282 5371 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:56:26.367573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38666 (local address 127.3.73.253:39573)
0807 10:56:26.367827 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:26.367837 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:26.368449 (+ 612us) server_negotiation.cc:407] Connection header received
0807 10:56:26.368727 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:26.368776 (+ 49us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:26.368986 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:26.369122 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:26.370011 (+ 889us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38666: BlockingRecv error: recv got EOF from 127.0.0.1:38666 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260807 10:56:26.393465 5371 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:56:26.390747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38670 (local address 127.3.73.253:39573)
0807 10:56:26.391018 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:26.391028 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:26.391714 (+ 686us) server_negotiation.cc:407] Connection header received
0807 10:56:26.391978 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:26.391982 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:26.392060 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:26.392176 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:26.393188 (+ 1012us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38670: BlockingRecv error: recv got EOF from 127.0.0.1:38670 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
W20260807 10:56:26.432279 5371 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:56:26.430052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38682 (local address 127.3.73.253:39573)
0807 10:56:26.430305 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:26.430312 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:26.430747 (+ 435us) server_negotiation.cc:407] Connection header received
0807 10:56:26.430927 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:26.430933 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:26.431046 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:26.431183 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:26.432010 (+ 827us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38682: BlockingRecv error: recv got EOF from 127.0.0.1:38682 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
W20260807 10:56:26.502995 5371 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:56:26.500780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38698 (local address 127.3.73.253:39573)
0807 10:56:26.501000 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:26.501006 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:26.501453 (+ 447us) server_negotiation.cc:407] Connection header received
0807 10:56:26.501641 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:26.501647 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:26.501752 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:26.501883 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:26.502741 (+ 858us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38698: BlockingRecv error: recv got EOF from 127.0.0.1:38698 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260807 10:56:26.637892 5371 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 10:56:26.635593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38702 (local address 127.3.73.253:39573)
0807 10:56:26.635790 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:26.635796 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:26.636231 (+ 435us) server_negotiation.cc:407] Connection header received
0807 10:56:26.636371 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:26.636377 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:26.636481 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:26.636617 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 10:56:26.637623 (+ 1006us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38702: BlockingRecv error: recv got EOF from 127.0.0.1:38702 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 10:56:26.650715 5391 catalog_manager.cc:1259] Loaded cluster ID: 05f1d971fdf540be98c78d55ed3028c5
I20260807 10:56:26.650837 5391 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c2352a20953f4d10b9fbb3009d16713b: loading cluster ID for follower catalog manager: success
I20260807 10:56:26.653884 5391 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c2352a20953f4d10b9fbb3009d16713b: acquiring CA information for follower catalog manager: success
I20260807 10:56:26.655951 5391 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c2352a20953f4d10b9fbb3009d16713b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 10:56:26.726982 5412 catalog_manager.cc:1259] Loaded cluster ID: 05f1d971fdf540be98c78d55ed3028c5
I20260807 10:56:26.727177 5412 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6f22e95eb2cd421caa9325952c2edda2: loading cluster ID for follower catalog manager: success
I20260807 10:56:26.730538 5412 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6f22e95eb2cd421caa9325952c2edda2: acquiring CA information for follower catalog manager: success
I20260807 10:56:26.733273 5412 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6f22e95eb2cd421caa9325952c2edda2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 10:56:26.905542 5371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:26.898376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38716 (local address 127.3.73.253:39573)
0807 10:56:26.898658 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:26.898663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:26.898789 (+ 126us) server_negotiation.cc:407] Connection header received
0807 10:56:26.899068 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:26.899073 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:26.899250 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:26.899370 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:56:26.899824 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:26.900623 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:26.902527 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:26.903026 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:26.903148 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:26.903968 (+ 820us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:56:26.904576 (+ 608us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:56:26.904993 (+ 417us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:26.905050 (+ 57us) server_negotiation.cc:299] Negotiation successful
0807 10:56:26.905165 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260807 10:56:26.919843 5558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:26.912503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52762 (local address 127.3.73.252:42165)
0807 10:56:26.913170 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:26.913180 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:26.913295 (+ 115us) server_negotiation.cc:407] Connection header received
0807 10:56:26.913633 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:26.913638 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:26.913821 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:26.914028 (+ 207us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:56:26.914528 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:26.915383 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:26.917202 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:26.917668 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:26.917850 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:26.918464 (+ 614us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:56:26.919081 (+ 617us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:56:26.919272 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:26.919381 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 10:56:26.919483 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":400,"threads_started":1}
I20260807 10:56:26.946976 5568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:26.927499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44668 (local address 127.3.73.254:37305)
0807 10:56:26.938912 (+ 11413us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:26.938922 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:26.938957 (+ 35us) server_negotiation.cc:407] Connection header received
0807 10:56:26.939143 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:26.939153 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:26.939463 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:26.939733 (+ 270us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:56:26.940919 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:26.942061 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:26.944292 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:26.944818 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:26.944991 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:26.945613 (+ 622us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:56:26.946124 (+ 511us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:56:26.946253 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:26.946439 (+ 186us) server_negotiation.cc:299] Negotiation successful
0807 10:56:26.946549 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11210,"thread_start_us":12393,"threads_started":1}
I20260807 10:56:26.948436 5558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:26.927882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52770 (local address 127.3.73.252:42165)
0807 10:56:26.928149 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:26.928159 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:26.940853 (+ 12694us) server_negotiation.cc:407] Connection header received
0807 10:56:26.941193 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:26.941199 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:26.941305 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:26.941419 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:56:26.942067 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:26.943196 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:26.945934 (+ 2738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:26.946717 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:26.946856 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:26.947512 (+ 656us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:56:26.947667 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:56:26.947779 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:26.948019 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 10:56:26.948117 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260807 10:56:26.949841 5371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 10:56:26.938207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38728 (local address 127.3.73.253:39573)
0807 10:56:26.938436 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 10:56:26.938443 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 10:56:26.941310 (+ 2867us) server_negotiation.cc:407] Connection header received
0807 10:56:26.941544 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 10:56:26.941550 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 10:56:26.941654 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 10:56:26.941752 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 10:56:26.942569 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:26.943475 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:26.945963 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 10:56:26.946662 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 10:56:26.946816 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 10:56:26.947764 (+ 948us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 10:56:26.947995 (+ 231us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 10:56:26.948136 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 10:56:26.949086 (+ 950us) server_negotiation.cc:299] Negotiation successful
0807 10:56:26.949293 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260807 10:56:26.958091 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 5038
I20260807 10:56:26.959293 5127 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:56:27.074575 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 5038
I20260807 10:56:27.107987 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 5145
I20260807 10:56:27.109313 5230 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:56:27.224576 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 5145
I20260807 10:56:27.258368 3367 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskRBZOut/build/asan/bin/kudu pid 5259
I20260807 10:56:27.259779 5355 generic_service.cc:196] Checking for leaks (request via RPC)
I20260807 10:56:27.373683 3367 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskRBZOut/build/asan/bin/kudu with pid 5259
2026-08-07T10:56:27Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5379 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10785 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (419516 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST